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PREFACE,  VOLUME  II  / 

/-ot+r ?|  / 

CO  i  C.C*.  ti'go  j 

This  volume  contains  notes  from  the  Workshops  that  were 
held  under  Contract  No.  MDA903-76-C-0207.  -^Section  I  is  the  set 
of  background  notes  on  Cu/ systems  that  was  distributed  to  all 
workshop  participants.  Sections  II  through  VII  contain  notes 
associated  with’^the  individual  workshops r  In  each  case,  the 
section  contains  the  following  items: 

a.  The  list  of  workshop  attendees 


b.  The  letter  of  invitation  that  was  sent  to 
invitees 

c.  A  set  of  questions  that  was  sent  to  invitees 
along  with  the  letter  of  invitation 

d.  A  set  of  notes  based  on  workshop  discussions 
that  was  distributed  to  attendees  following  each 
workshop. 
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SECTION  I 

BACKGROUND  NOTES  ON  C3  SYSTEMS: 
DISTRIBUTED  TO 
TO  ALL  WORKSHOP  PARTICIPANTS 
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Background  Notes  for  Workshops  on 
Human  Factors  in  Command,  Control  and  Communication  Systems 

R.  S.  Nickerson,  M.  J.  Adams,  and  R.  W.  Pew 

These  notes  are  intended  to  provide  background  material  for 

a  series  of  workshops  on  Human  Factors  in  Command,  Control  and 

Communication  (C3)'  Systems.  The  workshops  are  to  be  addressed 

3 

to  specific  topics  relating  to  human  factors  issues  in  C  system 

design  and  operation.  The  objectives  of  these  meetings  are  to 

consider  what  is  known,  within  the  area  delimited  by  the  meeting 

3 

topic,  that  has  important  implications  for  C  systems;  to 
identify  and  articulate  unanswered,  researchable ,  human  factors 
questions  that  relate  in  a  significant  way  to  the  effective 
functioning  of  these  systems;  and  to  interest  potential  contri¬ 
butors  to  a  research  program  in  this  problem  area  in  the  possibility 
of  becoming  involved. 

3 

1.  What  is  a  C  System? 

For  our  purposes  a  command,  control  and  communication  system 
is  a  system,  the  function  of  which  is  to  direct  the  activity  of 
some  organizational  or  operational  entity.  Such  systems  are  ob¬ 
viously  of  critical  importance  in  the  context  of  national  defense; 
they  are  also  applicable,  however,  to  a  wide  variety  of  non-defense 
problems  such  as  space  flight,  commercial  air-traffic  control, 
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law  enforcement,  disaster  relief  management,  and  industrial 

production.  The  emphasis  in  these  notes  is  on  defense-related 
3 

C  systems. 

Department  of  Defense  Directive  5100.30,  2  December,  1971, 
defines  command 'and  control  as  "the  exercise  of  authority  and 
direction  by  duly  designated  authorities."  It  goes  on  to  say 
that  "These  functions  are  performed  through  an  arrangement  of 
personnel,  equipment,  communications,  facilities  and  procedures 
which  are  employed  in  planning,  directing,  coordinating  and 
controlling  operational  activities  of  U.S.  military  forces" 
(Section  IIIB) . 

Although  one  typically  sees  the  terms  used  in  concert,  it  i 
important  to  recognize  that  command  and  control  are  not  the  same 
things.  Control  implies  a  two-way  communication  capability, 
whereas  command  does  not.  As  Edge  (1975)  points  out,  a  command 
system  need  not  have  a  feedback  loop,  but  a  control  system  must. 
One  can  issue  commands  "open  loop,"  and  under  some  circumstances 
this  may  be  a  reasonable  thing  to  do.  It  is  not  an  acceptable 
way  to  do  things,  however,  if  one  wants  to  be  able  to  respond 
effectively  to  the  dynamics  of  a  changing  situation.  A  command 
and  control  system  must  be  able  not  only  to  convey  command 
decisions  to  appropriate  action  units,  but  to  provide  for  the 
commander  accurate  and  timely  information  that  is  relevant  to 
the  mission  of  his  command,  and  this  information  must  include 
feedback  regarding  the  effects  of  commands  issued  in  the  past. 
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Thus,  the  simplest  representation  of  a  command  and  control 
system  would  show  communication  in  two  directions— from  command 
to  action  units  and  from  action  units  to  command.  In  fact,  the 
communications  requirements  of  modern  command  and  control  systems 
are  very  great  and  the  systems  that  are  being  developed  to  meet 
these  requirements  are  extremely  complex.  The  general  acceptance 
of  the  tri-partite  description  command,  control  and  communication 
is  acknowledgment  of  the  absolute  dependence  of  command  and 
control  on  adequate  communication  facilities. 

It  is  generally  recognized  that  computer  technology  also  has 
far  reaching  implications  for  the  military,  and  for  plans  and 
programs  relating  to  national  defense.  So  dependent  are  modern 
command  and  control  operations  on  computers  that  some  writers 
have  suggested  that  the  term  "C3  systems”  is  something  of  a  mis¬ 
nomer,  and  should  be  replaced  with  the  more  accurate  "C4  systems," 
the  fourth  C  obviously  standing  for  Computer  (Williams,  1975)  . 

Other  writers  have  noted  that  the  line  of  demarcation  between 
communications  functions  and  computer  functions  is  becoming 

increasingly  blurred  (Robbins,  1975) .  In  these  notes  we  retain 
3 

the  C  -system  designation  as  a  matter  of  convention  ,  but  we  recognize 
the  criticality  of  the  computer's  role  in  such  systems. 
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2.  Crisis  Management  and  Crisis  Prevention,  the  Dual  Roles 

'  3  " 

of  C  Systems 

One  role  of  military  C3  systems  is  to  direct  forces  during 
times  of  military  action.  Another  is  to  provide  decision  makers 
with  the  information  and  tools  that  can  help  them  anticipate, 
contain  and  solve  problems  before  they  erupt  into  military  con¬ 
frontations.  We  might  refer  to  one  of  these  functions  as  crisis 
management,  and  to  the  other  as  crisis  prevention. 

The  first  of  these  roles  is  perhaps  more  in  keeping  with 
prevailing  conceptions  of  what  military  command  and  control  is 
all  about  than  is  the  second.  And  the  importance  of  this  role 
clearly  is  great.  That  the  quality  of  a  C3  system  and  the  skill 
with  which  it  is  used  can  determine  the  effectiveness  of  a 
fighting  force  is  apparent,  and  has  prompted  the  use  of  such 
terms  as  "force  multiplier"  and  "multiplier  effect"  in  reference 
to  such  systems  and  to  the  consequences  of  their  utilization 
(Clements,  1975;  Michaelis,  1975;  Reed,  1975) . 

The  second  role  is  at  least  as  important  as  the  first, 
however.  Obviously,  the  more  effective  such  systems  are  at 
crisis  prevention,  the  less  need  there  will  be  for  crisis  manage¬ 
ment.  A  primary  goal  for  research  relating  to  C  systems  and 
operations  must  be  to  increase  the  capabilities  of  these  systems 
to  recognize  and  deal  with  incipient  problems  before  they  grow 


to  crisis  proportions. 
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3 .  The  World-Wide  Military  Command  and  Control  System  (WWMCCS) 

Several  systems  and  system-supporting  facilities  have  been 
developed  to  meet  U.S.  military  command  and  control  requirements. 

Of  overriding  importance  among  them,  for  at  least  the  near-term 
future,  is  the  WWMCCS .  By  DoD  Directive  (5100.30,  2  December, 

1971) ,  the  WWMCCS  is  "the  system  that  provides  the  means  for 
operational  direction  and  technical  administrative  support  involved 
in  the  function  of  command  and  control  of  U.S.  military  forces." 

In  conception,  the  WWMCCS  is  a  metasystem,  linking  together  and 
coordinating  the  activities  of  all  systems  that  perform  functions 
vital  to  U.S.  military  command  and  control.  More  specifically, 
the  WWMCCS  is  to  provide  "the  means  by  which  the  NCA  and  appropriate 
subordinates  may:  monitor  the  current  situation  to  include  the 
status  of  U.S.  and  non-U. S.  forces;  respond  to  warning  and  threat 
assessment;  employ  forces  and  execute  operational  plans;  perform 
attack,  strike  and  damage  assessment;  reconstitute  and  redirect 
forces;  and  terminate  hostilities  and  active  operations"  (Reed,  1975). 

3 . 1  Background 

The  WWMCCS  concept  was  first  established  by  DoD  Directive  in 
the  early  1960s.  The  intent  was  to  link  the  command  and  control 
systems  that  had  been,  or  were  being,  developed  by  various  commands 
to  a  national  command  and  control  system  that  would  support  the 
National  Command  Authorities  (NCA) ,  which,  at  the  time,  were 
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the  President,  the  Secretary  of  Defense,  and  the  Joint  Chiefs  of 
Staff.  The  Directive  specified,  or  at  least  allowed,  a  from-the- 
bottom-up  approach  to  the  problem.  The  Unified  and  Specified 
Commands  (e.g.,  European  Command,  Pacific  Command,  Strategic  Air 
Command)  were  authorized  to  develop  command  and  control  systems 
to  meet  the  particular  requirements  of  their  own  missions.  The 
systems  were  to  interface  so  as  also  to  meet  the  needs  of  the  NCA , 
but  the  emphasis  was  on  the  local  commands. 

The  approach  failed  to  produce  the  desired  results.  In 
particular,  communications  did  not  work  well,  especially  between 
echelons.  The  automated  data  processing  (ADP)  facilities  of  the 
various  systems,  which  had  been  developed  to  different  specifica¬ 
tions  and  implemented  with  different  equipment,  could  not  be  made 
to  function  smoothly  in  concert. 

A  rethinking  of  the  problem  led  to  the  abandonment  of  the 
from-the-bottom-up  approach  and  to  a  move  in  the  direction  of 
centralized  planning  and  control,  and  of  standardization  of 
equipment  and  procedures.  A  study  directed  in  1966  by  the 
Secretary  of  Defense  and  conducted  by  the  Joints  Chiefs  of  Staff 
( JCS )  resulted  in  the  decision  to  procure  35  computers  for  the 
WWMCCS  from  a  single  manufacturer  on  a  multiyear  basis  rather 
than  to  continue  to  let  each  command  acquire  its  own  equipment. 

A  contract  for  this  procurement  was  awarded  to  Honeywell  Informa¬ 
tion  Systems  in  1971.  The  equipment  was  to  be  all  off-the-shelf, 
nothing  still  being  researched  or  under  development  was  to  be 
included. 
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A  new  DoD  Directive  (5100.30,  2  December,  1971)  defined  the 
primary  function  of  the  WWMCCS  to  be  to  support  the  NCA  (which 
now  included  only  the  President  and  the  Secretary  of  Defense,  or 
their  duly  appointed  alternates  or  successors) .  It  identified 
the  National  Military  Command  System  (NMCS)  as  the  focal  point 
of  the  WWMCCS,  and  directed  that  command  and  control  systems  of 
all  other  DoD  components  be  configured  and  operated  for  effective 
support  of  it,  as  well  as  for  their  specific  missions.  In  par¬ 
ticular,  the  Directive  required  that  all  communications  facilities 
of  these  commands  be  compatible  with  those  of  the  NMCS.  A 
secondary  function  of  the  WWMCCS  is  to  support  the  command  and 
control  systems  of  the  Unified  and  Specified  Commands  and  WWMCCS- 
related  management/information  systems  of  other  DoD  components. 

The  emphasis  of  this  Directive  on  the  priority  of  the  NCA 
within  the  WWMCCS  mission  serves  two  purposes.  First,  it  mandates 
that  subordinate  commands  be  properly  interfaced  with  the  national 
command  so  as  to  be  maximally  responsive  to  national  priorities. 
Second,  it  provides  for  the  standardization  and  inter-operability 
that  are  necessary  to  support  flexible  and  dynamic  shifts  in  the 
delegation  and  exchange  of  command  and  control  between  national 
and  specific  command  levels  (Riceman,  1975) . 

3 . 2  Components 

The  WWMCCS  is  made  up  of  the  following  components:  The 
National  Command  System,  the  command  and  control  systems  of  the 
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Unified  and  Specified  Commands,  WWMCCS -related  management/ 
information  systems  of  the  Headquarters  of  the  Military  Depart¬ 
ments;  the  command  and  control  systems  of  the  Headquarters  of 
the  Service  Component  Commands,  the  command  and  control  support 
systems  of  DoD  Agencies,  and  several  non-DoD  systems. 

3.2.1  The  National  Military  Command  System  (NMCS) 

The  purpose  of  the  NMCS  is  to  provide  the  means  by  which 
the  primary  function  of  the  WWMCCS  is  to  be  realized.  Responsi¬ 
bility  for  the  establishment  and  implementation  of  operational 
policies  and  procedures  for  all  components  of  the  NMCS  resides 
with  the  chairman  of  the  Joint  Chiefs  of  Staff  under  the  direction 
of  the  Secretary  of  Defense. 

The  NMCS  is  made  up  of  the  following  components: 

-  The  National  Military  Command  Center  (NMCC) :  created  in 
1965,  located  in  the  Pentagon,  contains  Moscow  "hotline,"  General 
or  flag  officer  present  at  all  times. 

-  The  Alternate  National  Military  Command  Center  (ANMCC) : 
located  at  Fort  Ritchie,  Maryland. 

-  The  National  Emergency  Airborne  Command  Post  (NEACP) :  an 
airplane  at  Andrews  Air  Force  Base,  Maryland,  equipped  to  function 
as  the  NMCC  in  the  event  that  both  the  NMCC  and  the  ANMCC  are 
disabled . 

-  Other  Command  and  Control  Facilities  as  may  be  designated 
by  the  Secretary  of  Defense. 
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A  significant  part  of  the  NMCS  is  the  communication  linkages 
among  the  various  centers,  and  between  the  centers  and  other 
command  facilities.  Each  of  the  centers  must  be  ready  for  opera¬ 
tion  at  all  times. 


3.2.2  Command  and  Control  Systems  of  the  Unified  and  Specified 
Commands  (including  command  and  control  systems  of  sub¬ 
ordinate  unified  commands  and  joint  task  forces) 

-  Atlantic  Command 

-  European  Command 

-  Pacific  Command 

U.S.  Forces,  Korea 
U.S.  Forces,  Japan 
Taiwan  Defense  Command 

Military  Assistance  Command,  Thailand 

-  Strategic  Air  Command 

-  Readiness  Command 

-  Continental  Air  Defense  Command 


3.2.3  WWMCCS-Related  Management/Information  Systems  of 
the  Headquarters  of  the  Military  Departments 

-  Headquarters,  Department  of  the  Army 

-  Headquarters,  Department  of  the  Air  Force 

-  Chief  of  Naval  Operations 

-  Headquarters,  U.S.  Marine  Corps 
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3.2.4  Command  and  Control  Systems  of  Headquarters  of  the 
Service  Component  Commands 

-  Military  Airlift  Command 

-  Military  Sealift  Command 

-  Military  Traffic  Management  Command 

3.2.5  Command  and  Control  Support  Systems  of  DoD  Agencies 

-  Defense  Civil  Preparedness  Agency 

-  Defense  Communications  Agency 

-  Defense  Intelligence  Agency 

-  Defense  Mapping  Agency 

-  Defense  Nuclear  Agency 

-  Defense  Supply  Agency 

-  National  Security  Agency 

3.2.6  Non-DoD  Systems 

The  Directive  calls  for  the  establishment  and  maintenance 
of  coordination  and  liaison  with  U.S.  government  non-DoD  systems 
that  have  functions  associated  with  the  NMCS.  Such  systems  or 
activities  include: 

-  White  House  Situation  Room 

-  State  Department  Operations  Center 

-  CIA  Indications  Office 

-  U.S.  Intelligence  Board  National  Indications  Center 

-  U.N.  Military  Mission 
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-  Office  of  Emergency  Preparedness  National  Warning  Center 

-  U.S.  Coast  Guard  Operations  Center 

-  FAA  Executive  Communications  Control  Center 

-  Other  designated  entities 

The  NMCS  is  to  receive  from  these  systems  protocol,  intelli¬ 
gence,  diplomatic,  and  economic  information,  and  to  provide  them 

with  military  information. 

The  WWMCCS  receives  surveillance  and  early  warning  informatio 

from  the  following  systems: 

-  The  Defense  Support  Program 

-  The  Ballistic  Early  Warning  System 

-  over-the-Horizon  Forward  Scatter  Radar  System 

-  SLBM  Detection  and  Warning  System 

Other  systems  and  facilities  may  become  part  of  the  WWMCCS  on 
a  temporary  basis  during  times  of  emergency  or  national  crisis. 

3.3  Responsibilities  and  Policy  Guidance 

3.3.1  Director,  Telecommunications,  Command  and  Control  Systems 

Primary  staff  responsibility  for  the  WWMCCS  and  WWMCCS-related 
systems,  excepting  responsibilities  mentioned  in  the  following 
paragraphs.  The  Directive  originally  assigned  this  responsibility 
to  the  Assistant  to  the  Secretary  of  Defense  (Telecommunications) ; 
but  this  position  was  abolished  by  Secretary  of  Defense  Schlesinger 
in  January  1974,  and  replaced  by  that  of  Director,  Telecommunications 

Command  and  Control  Systems. 
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3.3.2  Assistant  Secretary  of  Defense  (Intelligence): 

Primary  staff  responsibility  for  intelligence  collection 

and  reporting  systems. 

3.3.3  Assistant  Secretary  of  Defense  (Comptroller): 

Responsible  for  ADP  procurement. 

3.3.4  Chairman,  Joint  Chiefs  of  Staff: 

Responsible  for  operation  of  the  NMCS  (through  the  Deputy 
Director  J-3,  for  command  and  control,  a  U.S.  Air  Force  General). 

3.3.5  WWMCCS  Council: 

Provides  policy  guidance  for  development  and  operation  of  the 
WWMCCS,  and  is  responsible  for  performance  evaluation.  Members 
of  the  Council: 

-  Deputy  Secretary  of  Defense  (Chairman) 

-  Chairman,  Joint  Chiefs  of  Staff 

-  Assistant  Secretary  of  Defense  (Intelligence) 

-  Director,  Telecommunications,  Command  and  Control  Systems 
(See  3.3.1.) 

3.4  Equipment  and  Supporting  Software 

Thirty-five  off-the-shelf  Honeywell  6000  Computer  Systems. 

The  World-Wide  Data  Management  System.  Developed  under 
contract  by  Honeywell.  (Emphasis  on  standardization  of  both 
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hardware  and  software) . 

Plans  call  for  connecting  all  of  the  WWMCCS  computers  in 
a  network. 

3 .  b  Technical  Support 

Provided  by  the  Joint  Technical  Support  Activity,  a  field 
activity  of  the  DCA.  ADP  training  and  logistic  support  is  the 
responsibility  of  the  Air  Force. 

4.  Focus  of  C3  Research  and  Development 

To  be  effective,  a  C3  system  must  provide  for  the  aggregation 
and  flow  of  accurate,  timely,  useful  information,  both  vertically 
and  horizontally  within  the  command  and  control  structure.  The 
system  must  provide  the  means  by  which  the  commander  can:  (1) 
obtain  the  information  he  needs  to  arrive  at  decisions;  (2)  dis¬ 
seminate  commands  to  appropriate  units;  and  (3)  monitor  the  outcomes 
of  those  commands.  Similarly,  the  system  must  give  units  in  the 
field  the  means  for  coordinated  implementation  of  commands  and 
assessment  of  their  effects.  A  major  emphasis  in  C  research  and 
development,  therefore,  has  been  on  the  improvement  of  capabilities 
for  the  acquisition,  integration,  and  exchange  of  information 
between  command  posts. 

Some  of  the  systems  and  subsystems  that  have  resulted  from 
the  effort  to  meet  these  needs  are  listed  in  Table  1.  These  systems 
are  representative  of  those  comprising  the  WWMCCS  network.  The 
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responsibility  for  planning  and  approving  the  acquisition  of  major 
capabilities  and  for  developing  an  architecture  for  the  over 
all  WWMCCS  configuration  has  been  given  to  the  WWMCCS  Council. 

This  is  clearly  a  demanding  task.  The  problems  inherent  in  the 
development  of  a  total  system  configuration  of  this  scope  are  com¬ 
pounded  by  the  fact  that  the  system  must  be  able  to  accommodate 
unpredictable  events  as  well  as  any  changes  in  national  defense 
policy.  In  addition,  careful  consideration  must  be  given  to  the 
issues  of  cost-effectiveness  and  budgetary  constraints.  It  is  not 
yet  clear  what  the  configuration  of  the  eventual  WWMCCS  will  be; 
the  functional  goals  of  the  development  effort  may  be  expressed 
in  terms  of  certain  qualities  that  are  often  mentioned  as  WWMCCS 
requirements.  Among  these  are  the  following:  flexibility,  re¬ 
sponsiveness,  security,  survivability,  credibility  and  inter¬ 
operability  . 

4.1  Flexibility 

The  emphasis  on  flexibility  directly  reflects  recent  changes 
in  national  defense  policy.  Because  of  nuclear  proliferation  and 
the  changing  distribution  of  power,  the  doctrine  of  massive  re¬ 
taliation  to  a  major  act  of  aggression  against  the  United  States 
is  giving  way  to  one  of  precise,  flexible,  controlled  response 
(Clements ,  1975)  A  secondary  motivation  for  the  development  of 
a  flexible  C3  system  derives  from  the  need  for  cost-effective 
strategic  capabilities.  The  development  of  comprehensive, 
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reliable,  real-time  communications  and  ADP  facilities  should 
provide  command  authorities  with  the  ability  to  take  greater 
advantage  of  the  full  range  of  options  open  to  them. 

4.2  Responsiveness 

3 

responsiveness  of  a  C  system  depends  on  its  capacity 
j-q  provide  the  necessary  support  for  the  formulation  and  imple 
mentation  of  plans  within  the  time  frame  required  for  strategic 
or  tactical  impact.  Research  and  development  relating  to  the 
responsiveness  of  C"^  systems  is  focused  on  two  areas.  First, 
with  the  goal  of  assuring  that  the  commander  will  have  access  to 
the  information  he  needs,  and  can  disseminate  his  orders  within 
the  prevailing  time  constraints,  attention  is  being  directed 
towards  improvements  in  communications  technology.  Second,  with 
the  goal  of  assuring  that  the  commander's  decisions  will  be  based 
on  the  best  possible  information,  research  is  being  directed  toward 
determining  exactly  what  that  information  should  include,  and  how 
it  should  be  obtained,  analyzed,  and  presented.  It  is  here  that 
interdependence  of  communication  and  computer  technology  is 
most  clearly  seen. 

A  wide  range  of  research  activities  is  motivated  by  the  need 
to  minimize  the  time  required  to  obtain,  process,  transmit  and  use 
information.  Communications  efforts  include  the  development  of 
ultrareliable  combat  radios  and  SHF  satellite  links.  Throughout 
the  WWMCCS  tnere  is  a  need  for  automated  message  processing, 
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switching,  distribution,  and  log-keeping.  Much  effort  is  directed 
toward  minimizing  the  "writer-to-reader "  time.  There  is  also  an 
interest  in  minimizing  the  time  for  writing,  or  reading,  itself; 
an  interest  that  is  reflected  in  the  investment  in  voice,  video, 
graphic,  and  multimedia  communications  channels  and  display  systems. 

Because  of  the  volume  and  complexity  of  the  information  to 
be  transmitted,  emphasis  is  also  being  placed  on  the  development  of 
computer -to -computer  data  exchange  techniques.  This  effort  is 
illustrated  by  the  Prototype  WWMCCS  Intercomputer  Network  (PWIN) . 

The  ultimate  goal  of  the  computer  network  is  to  yield  quick  access 
to  multisource  information.  This  goal  raises  such  research  problems 
as:  how  should  one  construct  a  dynamic,  event-driven  data  base; 

when  should  one  compromise  updating  fur  efficient  data  manipulation; 
and  how  should  the  system  be  organized  so  as  to  be  both  useful  and 

accessible,  as  appropriate,  to  all  levels  within  the  command  and 
control  network? 

4.3  Security 

The  story  of  the  acquisition  by  the  British  of  the  German 
Enigma  machine  in  1939  and  the  breaking  of  the  cypher  in  which  the 
German  military  command  transmitted  its  most  sensitive  information 
and  secret  orders  is  only  beginning  to  come  to  light  (Winterbotham, 
1974).  Th  ->  criticality  of  this  event  in  determining  the  outcome 
of  World  War  II  will  undoubtedly  be  debated  by  historians  for  some 
time  to  come.  There  can  be  little  doubt,  however,  that  the  intel¬ 
ligence  gained  by  the  Allies  by  virtue  of  their  unsuspected 
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knowledge  of  many  of  the  enemy's  most  guarded  plans  and  intentions 
was  of  inestimable  value  in  the  making  of  many  of  the  decisions 
on  which  the  general  outcome  depended.  The  lesson  of  Enigma  is 
that  communications  are  of  doubtful  worth  unless  they  can  be 
secured  from  interception  by  hostile  forces. 

Primarily  for  reasons  of  security,  military  communications 
systems  have  typically  made  use  of  some  sort  of  non-voice  trans¬ 
mission  that  could  be  readily  encrypted.  Non-voice  methods  of 
person-to-person  communication  are  widely  believed  to  be  inefficient 
as  compared  to  voice,  however,  and  this  belief  is  beginning  to  be 
substantiated  by  data  obtained  in  laboratory  experiments  (Chapanis, 
1971,  1973;  Ochsman  &  Chapanis,  1974) .  The  need  for  secure  voice 

3 

communications  for  use  in  C  systems  has  been  recognized  by  many 
military  writers  (Albright,  1975;  Reed,  1975;  Williams,  1975). 
Considerable  effort  is  currently  being  directed  toward  the  develop¬ 
ment  of  methods  of  speech  encoding  that  will  make  digital  trans¬ 
mission — and,  consequently,  end-to-end  encrypton  of  speech 
practically  feasible.  Much  progress  has  been  made  on  this  problem; 
it  is  now  possible  to  transmit  speech  that  is  highly  intelligible 
at  around  1000  to  2000  bits  per  second,  and  the  bit  rate  is  likely 
to  be  reduced  further.  The  resulting  speech  may  not  sound  like 
that  of  the  speaker,  however,  and  it  remains  to  be  seen  what 
problems  will  be  encountered  when  an  attempt  is  made  to  introduce 
it  into  operational  systems. 
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4.4  Survivability 

The  triple  redundancy  represented  by  the  NMCS,  the  ANMCC 
and  the  NEABCP  illustrates  the  concern  for  the  importance  of 
having  a  National  Military  Command  System  that  can  survive  a 
massive  military  attack  on  the  United  States.  The  Advanced  Air¬ 
borne  National  Command  Post  (AABNCP ) ,  a  planned  replacement  for 
the  NEABCP,  represents  an  attempt  to  increase  the  survivability 
of  C3  operations  still  further.  The  AABNCP  will  differ  from  the 
NEABPC  in  several  respects,  among  them  the  following:  increased 
communications  capability  (e.g.  ,  SIIF  satellite  communications 
terminal) ,  more  effective  protection  from  electronic  counter¬ 
measures,  greater  endurance,  and  a  larger  battle  staff  area. 

4.5  Credibility 

The  point  of  developing  systems  such  as  the  WWMCcS  is  to 
provide  decision  makers  with  the  information  and  tools  that  will 
permit  them  to  assess  situations  accurately  and  to  make  the  best 
possible  choices  among  the  decision  alternatives  that  are  open 
to  them.  How  effectively  an  individual  uses  the  information  and 
capabilities  that  such  a  system  provides  will  depend,  in  part,  on 
the  degree  of  credence  that  he  gives  to  the  information  it  supplies 
and  the  amount  of  confidence  he  has  in  its  abi-i-ity  to  perform  as 
advertised. 

While  credibility  must  depend,  in  the  long  run,  on  demonstrated 
performance,  it  may,  initially,  be  influenced  by  many  factors  in 
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addition  to  the  intrinsic  merits  or  limitations  of  a  system. 

Some  insight  into  this  problem  is  provided  by  a  study  conducted 
by  Morton  (1962)  in  which  four  retired  Air  Force  and  Navy  senior 
officers  (three  of  four-star  and  one  of  two-star  rank)  ware 
interviewed  in  an  effort  to  obtain  a  better  understanding  of 
how  commanders  perceive  their  jobs  and  how  they  relate  to 
electronic  command  and  control  systems.  While  all  of  the  inter¬ 
viewees  expressed  positive  attitudes  toward  technological  innova¬ 
tion  in  general,  they  volunteered  several  caveats.  Some  skepticism 
was  expressed  regarding  the  ability  of  operating  forces  to  maintain 
and  utilize  some  of  the  equipment  found  in  electronic  command  and 
control  systems.  The  problem  of  backup  was  noted:  after  building 
up  a  dependence  on  a  sophisticated  system,  what  happens  if  the 
system  fails  and  it  is  necessary  to  fall  back  on  manual  procedures? 
Lack  of  credence  in  system  outputs  was  mentioned,  but  this  was 
perceived  to  be  a  problem  that  could  be  counteracted,  at  least 
in  part,  by  providing  commanders  with  more  adequate  instruction 
and  training  with  respect  to  the  systems  they  are  expected  to  use. 

A  related,  but  more  complex  problem,  is  that  of  the  fear  of 
erosion  of  authority.  One  interviewee  pointed  out  that  because 
commanders  tend  to  be  generalists,  they  may  lack  the  specialized 
technical  knowledge  to  understand  fully  the  algorithms  on  which 
a  computer-based  system's  operation  depends,  and  thus  feel  power¬ 
less  to  influence  the  rules  for  system  operation.  Such  observations 
underscore  the  need  for  careful  attention  to  a  variety  of  human 
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factors  problems,  and,  in  particular,  to  the  task  of  developing 
man-computer  dialog  techniques  that  will  permit  a  commander  to 
interact  with  a  system  in  ways  that  do  not  require  a  technical 
training  to  understand  what  is  going  on. 

4.6  Interoperability 

DoD  Directive  5100.30  specifically  mandates  the  standardi¬ 
zation,  compatibility  and  interoperability  of  the  subsystems 
comprising  the  WWMCCS .  One  evidence  of  this  emphasis  is  the 
designation  of  a  specific  computer  system,  the  Honeywell  6000, 
as  the  standard  machine  for  the  WWMCCS  ADP  requirements.  Another 
move  towards  standardization  was  the  decision  that  one  standard 
terminal  system  would  be  specified  for  AUTODIN  use  in  the 
Eighties,  and  a  standard,  interim  terminal  system— the  Standard 
Remote  Terminal  (SRT) — for  the  Seventies.  Still  another  was  the 
adoption  of  a  standard  system — the  Automated  Multi-Media  Exchange 
(AMME) — for  use  by  Army  and  Navy  telecommunications  centers  with 
large  traffic  requirements  (Clements,  1975) .  The  ultimate  goal 
is  the  implementation  of  standard  equipment,  data  management 
systems,  programs,  terminology  and  formats  throughout  the  WWMCCS 
structure.  Thus,  the  specification  and  procurement  of  hardware 
and  software  that  can  function  in  a  variety  of  environments  is  a 
focal  problem  for  research  and  development.  There  also  are 
important  design  problems  relating  to  the  identification  of  those 
points  in  the  system  where  differences  are  necessary,  where  the 
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advantages  of  interoperability  are  outweighted  by  the  costs,  or 
where  security,  survivability,  or  other  desiderata  indicate  the 
utility  of  dedicated,  as  opposed  to  common,  channels. 

Several  new  programs  and  organizational  entities  have  been 
established  to  work  towards  insuring  interoperability.  A  case 
in  point  is  TRI-TAC,  an  organization  that  was  created  for  the 
express  purpose  of  providing  an  integrated  approach  toward  the 
development  of  tactical  systems  for  the  various  military  services. 
The  goals  of  TRI-TAC,  according  to  Reed  (1975,  p.  9),  are,  "(1) 
to  achieve  the  necessary  degree  of  interoperability  among  U.S. 
tactical  communications  systems  and  to  resolve  the  interface  with 
DCS,  NATO  and  other  national  systems  as  required;  (2)  to  place  in 
the  field  in  a  timely  manner  newly  developed  tactical  communica¬ 
tions  equipment  to  efficiently  meet  the  needs  of  joint  and  service- 
unique  missions;  and  (3)  to  eliminate  unnecessary  duplication  of 
service-unique  equipment." 

3 

5 .  Human  Factors  m  C  Systems 

3  .  . 

The  fundamental  purpose  of  a  C  system  is  to  optimize  the 

quality  of  command  decisions  and  the  timeliness  with  which  they 

are  made  and  communicated.  From  a  human  factors  point  of  view, 

3 

therefore,  two  fundamental  questions  to  consider  about  any  C 
system  at:  any  level  of  functioning  are  these:  (1)  What  information 
should  be  presented  to  the  users,  and  (2)  How  should  that  informa¬ 
tion  be  represented?  The  first  question  is  one  of  content:  the 
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second,  one  of  form.  These  questions  give  rise  naturally  to  a 
host  of  others  that  lead  one  into  virtually  every  major  area  of 
psychological  research:  perception,  attention,  language,  memory, 
decision  makrng,  problem  solving,  motivation.  Indeed,  the  basic 
human  factors  problem  in  C3  systems  is  not  so  much  that  of 
identifying  what  areas  of  psychological  research  are  relevant- 
because  most  of  them  are— but  that  of  determining  what  specific 
research  findings  (including  potential  findings)  could  have  truly 
significant  impact  on  the  degree  to  which  these  systems  meet  the 

needs  of  their  users. 


The  importance  of  this  focus  on  the  users'  needs  is  recognized 
by  people  with  developmental  or  operational  responsibilities  for 
C3  systems.  Clements  (1975),  for  example,  points  out  the  necessity 
for  attempting  to  determine  exactly  what  information  the  National 
Command  Authorities  will  require  in  order  to  deal  effectively  with 
specific  potential  situations.  Clements  also  has  challenged  the 
notion  that  the  quality  of  a  decision  is  likely  to  be  the  better 
the  greater  the  amount  of  relevant  information  the  decision  maker 
has  at  his  disposal,  by  noting  the  possibility  of  "information 
overload,"  and  warning  against  giving  commanders  more  information 
than  they  can  digest  in  a  fast  moving  crisis.  That  more  information- 
even  when  it  is  highly  relevant  to  the  decision  maker's  task— does 
not  .  i .  iably  lead  to  better  decisions  has  been  demonstrated  m 
laboratory  experiments  (Hayes,  1964)  . 
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The  evidence  that  people  are  rather  severely  limited  in  their 
ability  to  apply  many  disparate  items  of  information  effectively 
to  decision-making  problems  illustrates  the  interdependence  of  the 
"what"  and  "how"  aspects  of  information  presentation.  The  problem 
of  "information  overload"  may  be  less  one  of  amount  of  information 
and  more  a  question  of  packaging.  Even  a  small  amount  of  informa¬ 
tion  may  exceed  the  human  being's  capacity  to  make  effective  use 
of  it  if  it  is  presented  in  suboptimal  ways;  conversely,  if  it  is 
properly  represented,  one  may  be  able  to  deal  effectively  with 
very  large  quantities  of  it.  One  of  the  goals  of  developers  of 
C  systems  is  to  increase  the  capabilities  of  these  systems  to 
analyze,  correlate  and  integrate  data  and  to  present  the  results 

to  a  decision  maker  in  forms  that  are  well  suited  to  his  use 

(Michaelis,  iy75) . 

The  workshops  for  which  these  notes  are  intended  as  background 

material  will  focus  on  a  variety  of  research  areas.  The  goal  in 

each  case,  however,  will  be  to  determine  what  research  should  be 

.  3 

done  in  order  to  maximize  the  chances  that  C  systems  will,  in 

fact,  meet  user  needs. 
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Table  1.  Representative  systems  and  system-supporting 

3 

facilities  that  relate  to  C  operations 

Department  of  the  Army  Command  and  Control  System  (DACCS) : 

The  Army  subsystem  that  supports  the  WWMCCS.  Its  center, 
the  Army  Operations  Center  (AOC)  is  located  in  the  Pentagon. 

Defense  Communications  System  (DCS) :  RDT&E  program  involves 
10  areas:  switching,  transmission,  terminals,  secure  commu¬ 
nication,  satellite  subsystem  (DSCS) ,  system  control, 
survivability  analysis,  operations  research/model  development, 
network  design,  and  system  integration  and  transition 
validation  (Hoversten,  197  5)  . 

Emergency  Message  Automatic  Transmission  System  (EMATS) : 

System  by  which  National  Command  Authorities  communicate 
with  CINCs  in  times  of  crisis. 


Fleet  Command  Center  (FCC) :  Intended  to  be  for  Fleet  Commander 
what  TFCC  is  for  Ship  Commander,  and  also  to  provide  interface 
between  TFCC  and  all  other  data  sources. 
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Joint  Tactical  Information  Distribution  System  (J-TIDS) : 

Developmental  program  under  Air  Force  management,  ultimately 
expected  to  develop  a  set  of  communications  terminals  that 
will  satisfy  the  requirements  of  various  military  platforms. 

Military  Airlift  Command  Integrated  Management  System  (MACIMS) : 
System  linking  computers  at  six  Aerial  Ports  of  Embarkation 
to  computers  at  Military  Airlift  Command  Headquarters  at 
Scott  AFB ,  Illinois.  One  hundred  and  thirty-four  remote 
terminals  planned;  over  100  currently  in  operation. 

Minimum  Essential  Emergency  Communications  Net  (MEECN) :  The 
portion  of  the  WWMCCS  that  is  considered  survivable  of  a 
heavy  nuclear  attack  on  the  United  States,  and  equipped  to 
send  command  messages  to  nuclear  forces.  Development  is  the 
responsibility  of  the  MEECN  Systems  Engineer  of  the  DCA. 

Modular  Automated  Communication  System  (NAV-MACS) :  System  being 
developed  (at  several  levels)  by  Naval  Electronic  Systems 
Command  to  automate  such  functions  as  message  processing, 
switching,  reproduction,  distribution  and  record  keeping. 

Built  for  shipboard  use  around  the  AN/UYK-20. 
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Prototype  WWMCCS  Intercomputer  Network  (PWIN) :  Mini-computer 
network  being  developed  by  JTSA  for  the  purpose  of  studying 
internetting  requirements  of  WWMCCS ,  and  for  testing  software. 

Remote  Job  Entry  Terminal  System  (RJETS ) :  Air  Force  system 
providing  remote  access  to  Burroughs  3500s  and  UNIVAC  1050s 
located  at  host  bases.  RJETS  are  mini-computers;  180  such 
terminal  systems  are  planned. 

Shipboard  Data  Multiplex  System  (SDMS) :  General-purpose 

information  transfer  system  being  developed  by  the  Naval 
Sea  Systems  Command  for  internal  data,  voice  and  video 
intership  communication  requirements. 

Tactical  Flag  Command  Center  (TFCC) :  An  on-ship  center  (under 
development)  that  is  intended  to  receive  data  from  a  variety 
of  sources  (e.g.,  Ships  Signal  Exploitation  System,  Naval 
Tactical  Data  System,  Integrated  Operational  Intelligence  Center, 
Carrier  Tactical  Support  Center,  Ocean  Surveillance  Systems, 

Task  Force  Sensors)  correlate  them,  and  present  results  to  the 
commander.  Plans  call  for  exploitation  of  advanced  computer  and 
display  technology. 
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Wc  are  currently  engaged,  in  a  project  for  the  Human  Resources 
Office  of  the  Defense  Advanced  Research  Projects  Agency  on 
the  topic  of  Human  Factors  in  Command,  Control  and  Communi¬ 
cation  (C3)  systems.  A  major  objective  of  the  project  .is  the 
development  of  a  plan  for  a  program  of  research  on  this  topic. 

It.  is  not  our  purpose  to  design  C.s  systems,  but,  rather,  to 
iden t i f y~”r e sea r c ha b  1  e  problems  relating  to  the  performance 
of  C3  systems,  and  in  particular  to  that  of  the  users  of  those 

systems . 

As  one  means  to  this  end,  we  are  convening  a  series  of  work¬ 
shops  to  bring  together  small  groups  of  appropriate  individuals 
to  discuss  various  aspects  of  the  problem.  The  first  such 
workshop  will  be  held  in  Arlington,  Vi'  jinia  (BBN  office  a^ 

1701  North  Fort  Myer  Drive)  on  April  7,8,  1976.  The  purpose  of 
this  meeting  will  be  to  develop  a  better  understanding  of  C 
systems,  and  of  the  types  of  human  factors  issues  that  have  been, 
or  are  likely  to  be,  encountered  in  their  design  and  use. 

The  meeting  will  be  relatively  informal.  What  we  want  is  a 
far-ranging  discussion  of  issues  that  participants  consider  to 
be  germane  to  the  general  topic.  To  provide  a  context  and 
some  points  of  departure,  we  have  drafted  a  brief  background 
paper  on  C3  systems,  and  prepared  a  list  of  questions  that  are 
representative  of  those  that  we  would  want  to  cover  at  the 
meeting.  Both  of  these  documents  are  enclosed. 

My  reason  for  writing  you  is  to  .invite  you  to  participate  in 
this  meeting.  Our  plan  is  to  limit  the  number  of  participants 
to  approximately  12  to  15,  so  we  can  maintain  a  lively  discussion 
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in  an  informal  atmosphere.  We  have  tried  to  select  a  group  that 
includes  some  people  who  have  had  experience  with  systems 
as  either  researchers  or  users,  and  others  who  have  done  human 
factors  work  that,  while  not  explicitly  addressed  to  C  problems, 
seems  especially  relevant  to  them  in  some  way. 


Any  additions  you  might  like  to  make  to  the  questions  listed 
on  Enclosure  2  would  be  most  welcome.  If  you  mail  them  to  me, 
I  will  se°  that  they  get  incorporated  in  an  amended  list  that 
will  be  distributed  at  the  meeting.  Alternatively,  you  could 
brinq  them  to  the  meeting  and  we  can  amend  the  list  on  the  fly 
Of  course,  my  preference  is  for  the  first  option,  but,  being  a 
realist,  I  will  settle  for  the  second. 


If  you  would  like  to  participate  in  this  meeting  please  let 
me  know  as  scon  as  possible  by  returning  the  enclosed  postcard, 
and  I  will  crive  you  details  concerning  place  and  time.  We  will 
of  course,  reimburse  you  for  reasonable  expenses  incurred  as  a 
result  of  your  participation  (travel,  meals,  lodging)  ana  can 
offer  vou  an  honorarium  of  $150  per  day.  Perhaps  the  more 
significant  motivation  for  participation  is  the  chance  to  help 
shape  a  new  DARPA  urogram  that  is  likely  to  provide  research 
opportunities  for  human  factors  specialists  in  the  future. 

I  sincerely  hope  that  you  will  be  able  to  join  us  on  April  /. 

In  any  case,  I  would  appreciate  an  early  reply ,  because  ii  you 
are  unable  to  attend,  we  want  to  extend  the  invitation  to 
another  potential  invitee. 


Raymc/nd  S.  Nickerson 
f  tm 


Enc.  Background  paper 
List  of  questions 
Self-addressed  stamped  postcard 
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Workshop  £1:  Toward  a  Behavioral  Model  of  a  C  System 

The  primary  purpose  of  this  project  is  to  develop  a  plan 

3 

for  a  program  of  research  on  human  factors  in  C  systems.  As  a 
first  step  toward  this  objective  we  wish  to  address  the  general 
question  of  how  best  to  conceptualize  a  C  system,  given  the 
intent  of  focusing  on  human  factors  issues  and  problems.  The 
following  specific  questions  are  representative  of  those  that 
we  think  should  be  discussed  at  our  first  workshop.  .If  you 
agree  to  participate  in  this  workshop,  we  invite  you  to  modify, 
and  add  to,  this  list  as  you  see  fit. 


3 

1.  Is  it  possible  to  develop  a  model  of  a  C  system  that  would 
be  general  enough  to  be  representative  of  a  variety  of  actual 
systems,  but  still  detailed  enough  to  have  some  practical  value? 


2.  If  it  is  possible  to  generate  such  a  model,  what  would  its 
major  structural  and  functional  components  be?  And  what  kinds 
of  measurements  cculd  be  made  to  validate,  or  invalidate,  it? 

3 

3.  If  such  a  model  is  not  a  possibility  (i.e.,  if  particular  C 
systems  are  sufficiently  diverse  to  preclude  being  represented 
by  the  same  formalism),  is  it  possible  to  identify  system  types 
that  are  sufficiently  similar  to  be  represented  by  a  common  model? 

4.  What  kinds  of  formalisms  are  best  suited  to  the  representation 
3 

->f  C  systems? 
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5.  What  non-military  C.3  systems  would  represent  worthwhile  objects 
of  study?  To  make  a  worthwhile  study,  a  system  should  probably  be 
dealing  with  a  relatively  complex  command  and  control  problem; 

and  it,  as  well  as  documentation  describing  it,  should  be  readily 
accessible  to  investigators. 

6.  What  techniques  are  best  suited  to  the  study  of  operational 
3 

C  systems? 

7.  What  behavioral  indices  are  most  likely  to  be  useful  for 
such  a  study?  To  be  useful  an  index  must  be  obtainable  and 
interpretable. 

8.  What  aspects  of  the  performance  of  C3  systems  are  most 
constrained  by  human  limitations,  or  vunerable  to  human  error? 

9.  How  does  one  determine  points  of  vulnerability  before 
consequential  errors  have  occurred? 

10.  What  methods  are  used,  or  could  be  used,  to  evaluate  the 

3 

overall  performance  of  a  C  system? 


11.  How  could  those  methods  be  improved? 

12.  How  important  is  it  for  a  user  of  a  computer-based  system 
to  have  a  basic  understanding  of  what  a  computer  is  and  how  .it 
operates? 
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1J.  What  are  the  major  human  factors  issues  relating  to  C 
organizations  and  operations? 

14.  What  sorts  of  taxonomies  (task,  decision,  message,  etc.) 

.  .  3 

would  be  most  helpful  for  describing  C  systems? 

3 

15.  In  what  significant  ways  do  non-military  C  systems 

.  .  3 

correspond  to  (differ  from)  military  C  systems? 

3 

16.  What  are  the  critical  communications  patterns  in  C 
systems? 

3 

17 .  where  are  the  degrees  of  freedom  in  C  operations?  Which 

of  the  operations  are  carried  out  in  accordance  with  prespecified 
inflexible  procedures,  and  which  require  initiative  and  discretion 
on  the  part  of  the  operator? 

3 

18.  What  are  the  classic  failure  modes  of  C  systems? 

19.  What  are  the  information  requirements  and  information 

3 

sources  for  C  systems? 

20.  What  known  human  factors  principles  are  applicable  (hive 
been  or  should  be  applied)  to  the  design  and  operation  of  C! 
systems? 
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21.  what  unsolved  human  factors  problems  are  most  urgent  for 
3 

C  systems? 

22.  What  manual  backup  provisions  are  typically  made  for  coping 
with  failure  of  electronic  components  of  C3  systems? 

23.  What  general  principles  guide  the  allocation  of  C3 
functions  to  men  and  machines? 

L4.  what  specific  C3  systems  would  make  the  most  reasonable  foci 
for  study,  given  the  goal  of  identifying  human  factors  problems, 
the  solutions  to  which  would  have  significant  impact  on  national 

defense? 

25.  How  does  one  tell  in  advance  whether  the  solution  to  a 
particular  problem  will  have  a  significant  or  an  inconsequential 
impact  on  overall  system  performance? 


4 


Bolt  Beianek  and  Newman  Ino. 


1 

NOTES  FROM  APRIL  7,8  MEETING  RE  HUMAN  FACTORS  IN  C3  SYSTEMS: 

3 

1.  The  cost  and  character  of  a  solution  to  a  C  problem 
will  be  a  determinant  of,  and/or  determined  by,  the  level  within 
the  DoD  command  structure  at  which  the  solution  is  to  be  applied. 

In  order  to  maximize  the  chances  of  extendability  of  findings 
to  different  levels,  one  might  intentionally  focus  on  problems 
that  tend  to  characterize  the  middle  of  the  hierarchy. 

?.  Some  C3  systems  that  might  provide  foci  for  study: 

TOS,  ASSIST,  WWMCCS,  AVJACS,  TFCC,  CS-3,  TACFIRE,  ATC-FAA,  MAT, 

NATO,  NORAD,  NTDS,  MTDS. 

3.  Comment  of  participant  re  TOS:  Major  problems  were 
related  to  information  overload  and  to  the  difficulty  of  coding 
incoming  information.  Coders  were  apparently  unable  to  organize 
information  effectively  for  system  input.  They  found  it  difficult 
to  abstract  for  fileab.ility,  retrievability ,  meaningfulness,  or 
to  anticipate  commanders'  questions. 

3 

4.  A  suggested  set  of  C  system  functions: 

a.  data-management  functions  (includes  question-answering 
capability) 

b.  aids  to  documentation,  and,  in  particular,  message  productior 

c.  aids  to  communication 

d.  aids  for  scheduling  and  resource  management 

e.  aids  for  conferencing 

f.  computerized  tools  for  doing  calculations 
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g.  aids  for  modeling  (to  permit  the  conducting  of  "what  if" 
types  of  simulations;  to  predict  the  consequences  of 
alternative  courses  of  action) 

h.  capabilities  for  training 

i.  thinking  aids  (including  aids  for  decision  making) 

Vie  should  think  about  thepossibility  of  modifying  this  list 
in  such  a  way  as  to  make  an  explicit  distinction  between  functions 
that  have  to  be  performed  and  methods  by  which  they  are,  or  could 
be,  performed.  The  functions  would  include,  for  example,  data 
management  (including  data  acquisition,  data  entry,  data  manipulate 
and  dissemination),  document  production,  communication,  scheduling, 
computation,  modeling,  training,  and  decision  making. 

With  respect  to  the  methods  by  which  these  functions  are  performed 
one  would  want  to  distinguish  at  least  unaided-manual,  automated, 
and  interactive.  In  considering  interactive  processes,  one  might 
want  to  distinguish  among  several  modes  on  the  basis  of  whether, 
or  the  degree  to  which,  the  man  or  the  machine  dominates  the 

interaction - 

5.  The  traditional  principles  of  war  include:  unity  of 
command,  mass  of  force,  surprise,  maneuverability  (flexibility) , 
logistics  and  intelligence.  Question:  To  what  extent  do  these 
principles  still  apply? 

6.  Another  set  of  concepts  in  terms  of  which  one  could 
conceptualize  C3  systems  are  those  that  we  have  used  with  reference 
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to  decision  making  and  training:  information  gathering, 
data  evaluation,  problem  structuring,  hypothesis  generation, 
hypothesis  evaluation,  preference  specification,  action 
selection  and  decision  evaluation. 

7 .  A  distinction  was  made  between  algorithms  for  suggesting 
commands  and  algorithms  for  filtering  commands.  This  strikes  me 
as  similar  in  some  respects  to  the  distinction  between  hypothesis 
generation  and  hypothesis  testing.  Perhaps  the  more  general  point 
is  the  need  for  generative  or  synthetic  processes  and  for 
derivative  or  analytical  processes.  Generative  processes  include 
processes  for  generating  hypotheses,  for  generating  models,  for 
imposing  structure,  for  inducing  principles,  for  explicating 
unstated  premises  in  elliptical  arguments,  etc. 

8  A  distinction  should  be  made  between  commands  that  specify 
goals  or  objectives  and  those  that  specify  procedures.  (Goal- 
specific  versus  procedure-specific  commands.)  In  this  regard 
it  is  of  interest  to  consider  what  are  the  bounds  on  the  behavior 
of  any  given  component  in  a  C  system.  Presumably,  m  some  cases, 
the  behavior  is  precisely  prescribed  by  the  application  of  doctrine 
to  specific  situations.  In  other  cases,  presumably,  there  is 
considerable  latitude  for  judgment  and  choice. 
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9.  One  might  argue  that  the  primary  goal  of  a  C  system  is 
control,  and  that  command  and  communication  are  means  to  that  end. 
According  to  this  view,  what  one  is  attempting  to  do  is  to  control 
some  aspect  of  the  world  space,  and  the  way  one  does  that  is  to 
transmit  a  seguence  of  commands  to  appropriate  action  units.  the 
effectiveness  of  a  C3  system  then  would  be  evaluated  strictly  in 
terms  of  the  degree  to  which  the  desired  control  was  realized. 
j?ygj^y thing  else  is  a  question  of  method .  (We  should  check,  the 
literature  on  control  systems  to  see  if  we  might  find  something 

useful  in  it  to  apply  to  the  problem  of  developing  a  conceptual 
3 

model  of  C  systems.) 

10  The  point  was  made  that  it  is  not  safe  to  assume  that 
there  is  an  equitable  match  between  the  way  in  which  respon- 
sibilities  are  distributed  within  C  systems  and  the  mechanisms  for 
reward  for  good  performance. 

11.  A  "minimal  model"  of  a  C3  system  would  include:  (1) 

topology  (representing  communication  structure)  ,  (2)  media,  (3) 

protocols  (message  formats) ,  and  (4)  a  reinforcement  scheme 
(motivational  structure) . 

12,  In  judging  the  success  or  failure  of  a  C  system  it 
may  be  necessary  to  ask:  success  or  failure  from  whose  point  of 
view?  The  notion  is  that  how  success  and  failure  are  defined  may 
depend  on  whether  one  is  looking  at  performance  from  above  or  from 
below.  Or,  what  might  appear  to  be  a  successful  functioning  of  a 
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system  from  a  relatively  local  point  of  view  might  actually  con¬ 
stitute  a  failure  from  a  more  global  perspective,  and  vice  versa. 

13.  Among  the  various  ways  in  which  a  system  can  fail  are 
the  following: 

a.  Messages  (commands)  fail  to  get  to  appropriate 
destinations. 

b.  Messages  are  misinterpreted  by  their  recipients. 

c.  Overloading  of  some  system  component:  too  many  messages 
received  by  a  given  recipient.  This  suggests  the  need 
for  batter  methods  of  message  classification  and 

prioritization. 

d.  Misinterpretation  of  incoming  data;  failure  to  recognize 
indicator  patterns. 

e.  Results  of  explicit  attempts  to  deceive. 

f.  Lack  of  flexibility;  inability  to  change  course  or  to 
delay  commitment. 

g.  Inability  to  respond  to  unanticipated  first-time 
situations,  (e.g.,  the  Pueblo  incident,  the  121  shootdown, 
the  Czechoslovakian  invasion) . 

h.  Lack  of  credibility  of  messages. 

i.  Inability  to  get  from  one  specific  point  to  another  in 
the  communication  system. 

j.  Precedence.  The  problem  of  a  push-down  list  that  never  pops 
all  the  way  up;  the  mismatch  of  upward  precedence  and 
downward  precedence  in  a  hierarchically  organized  system. 
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k.  The  problem  of  information  inundation. 

l.  Ineffective  or  inappropriate  reward  structure, 
motivating  factors  that  are  inconsistent  with  the 

system's  mission. 

14.  Although  voice  communication  has  many  advantages  in  a 
C3  system,  one  of  the  problems  with  it  at  the  moment  is  the 
inability  to  produce  from  it  a  record  of  the  communication. 
Individuals  who  have  to  act  upon  commands  often  want  or  need  some 
documentary  record  of  the  command. 

15.  A  factor  that  may  be  very  important  in  the  functioning 
of  any  C3  system  is  the  sfyle  of  operation  of  the  commander.  One 
important  dimension  on  which  commanders  may  differ  is  the  extent 
to  which  they  are  willing  to  delegate  decision-making  authority, 
and  in  the  ways . in  which  they  use  their  staffs. 

16.  Re  human  limitations  that  are  relevant  to  the  operation 
a 

of  C  systems: 

a.  People  allegedly  are  not  very  good  at  estimating 
compound  probabilities. 

b.  There  is  a  general  class  of  problems  that  might  be 
referred  to  as  data- transformat ion  problems.  These  have 
to  do  with  taking  data  that  come  into  the  system  in  a 
variety  of  forms  and  transforming  them  .so  thai-  they  can 

used  effectively  as  system  inputs.  This  might  mf  an, 
for  example,  classifying  observations  and  imposing  some  . 
structure  or  format  on  data.  . 
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c.  The  problem  of  inaccuracy  in  the  reporting  of 
observations. 

d.  The  problem  of  reluctance  on  the  part  of  people  to 
consider  more  than  a  relatively  small  set.of  alternatives 
that  may  be  open  to  them.  (Example  of  doctors  tending 

to  prescribe  from  a  small  set  of  favored  drugs. 

e.  The  problem  of  inadequate  coordination  between  G2  and 
G3.  The  problem  of  intelligence  not  always  knowing 
enough  about  our  own  situation  and  activity. 

f.  The  problem  of  actions  sometimes  appearing  absurd  at 
one  level,  but  reasonable  at  a  more  global  level. 

g.  The  alleged  dependence  of  the  quality  of  decision 
making  on  the  tactical  advantage  or  disadvantage  at 
which  the  decision  maker  finds  himself. 

IV .  Re  how  to  determine  'points  of  vulnerability  in  a  system 
before  consequential  errors  have  occurred:  One  approach  that  can 
be  taken  in  simulation  exercises  is  that  of  stressing  specific 
comoonents  of  a  system  and  observing  the  effect  of  such  stress  on 
overall  system  performance,  or  of  intentionally  introducing  bottle 
necks,  or  of  forcing  specific  components  to  malfunction.  One 
might  have  participants  in  an  exercise  whose  gob  it  is  to  attempt 
to  cabotage  system  performance. 
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18.  Some  previous  studies  of  C  systems: 

a.  ARI's  field  evaluation  of  7th  Army  DEVTOS. 

b.  The  Operational  Applications  Laboratory's  research 
program  on  threat  evaluation. 

c.  The  human  factor’s  study  of  the  SAGE  system. 

d.  A  recent  developmental  study  of  the  NORAD  system. 

19.  Another  approach  to  the  evaluation  of  system  performance 
is  the  debriefing  of  system  users  following  the  use  of  a  system 

in  a  crisis  situation. 

20.  The  problem  of  recovery  (reconstitution  of  the  command 
structure)  following  military  attack  was  noted  as  a  serious  one. 

21.  C3  systems  of  the  future  will  have  to  be  able  to  cope 
with  multi-party  conflicts. 

22.  Attention  has  to  be  given  to  the  problem  of  interfacing 
our  own  C3  system  with  those  of  our  allies  and  even  possibly,  to 
some  extent,  with  those  of  our  antagonists  or  potential  enemies. 
Kith  respect  to  the  latter  possibility,  there  may  be  great  utility 
in  having  antagonists  mutually  informed,  at  least  in  a  limited  way 
concerning  each  other’s  plans  and  activities.  It  is  probably  not 
the  case,  given  the  potentially  devastating  consequences  of  spasm 
reaction  to  a  presumed  threat,  that  surprise  is  always  a  thing 
for  which  one  should  strive. 
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23.  Some  human  factors  problems  in  C3  systems  : 

a.  Too  many  messages  (although  we  should  not  assume  that 
the  messages  a  commander  receives  include  all  those  that 

he  needs) . 

b.  Ambiguous  messages. 

c.  In  spite  of  the  fact  tht  most  decisions  that  must  be  made 
are  stereotyped  decisions,  they  still  prove  to  be 
difficult,  given  the  time  constraints  under  which  they 


must  be  made. 

d.  The  most  critical  information  for  a  commander,  namely 

geographic  information  (e.g.,  maps  and  overlays)  is  not 
being  provided  very  well. 

c.  Data  bases  are  inadequate.  They  tend  to  be  out  of  date 
and  to  contain  erroneous  data,  and  they  are  difficult 
to  vise,  i.e.,  it  is  difficult  to  find  anything  in  them, 
f.  There  is  no  effective  way  for  distributed  groups  to 
solve  problems  better  than  do  individuals. 


24.  One  approach  to  the 


be  that  of  specifying  in  some 


study  of  existing  C3  systems  would 
detail  what  particular  individuals 


in  such  systems  have  to  do. 

25.  Perhaps  a  critical  factor  in  the  operation  of  a  C 
system  is  the  commander's  cognitive  model  of  the  system.  A  question 
of  some  interest  is  that  of  the  extent  to  which  the  commander 
understands,  or  should  understand,  the  structure  and  operation  of 
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the  system  at  various  levels.  It  seems  fairly  clear  that  he  must 
understand  them  at  some  level,  but  it  is  not  clear  that  he  need 
understand  them  at  all  levels.  Is  the  commander  who  does  under¬ 
stand  the  system  at  all  levels  (including,  for  example,  the  details 
of  operation  of  individual  programs  in  a  computer-based  system) 
able  to  utilize  the  system  more  effectively  than  the  commander  who 
does  not  have  such  detailed  understanding? 


26.  A  distinction  was  made  between  two  types  of  adaptation 
required  of  a  system.  The  first  might  be  referred  to  as  tuning; 
in  this  case  the  system  adjusts  gracefully,  and  more  or  less 
continuously,  to  either  extrinsic  or  intrinsic  stimuli  for  change. 
The  second  type  of  adaptation  involves  quantum  jumps  from  one 
level  of  operation  to  another.  Such  jumps  are  necessary  sometimes 
in  order  to  respond  to  crisis  situations  or  unanticipated  develop¬ 
ments.  They  may  involve  changes  in  goals  and  priorities  as  well 

/ 

as  qualitative  changes  .in  operation .  j 

27.  An  impediment  to  maximally  effective  C  systems  can  be 
the  existence  of  conflicting  goals  at  different  levels  within  the 
system.  Such  conflicts  usually  do  not  result  m  the  explicit 
disobeying  of  orders,  but,  rather,  in  creative  insubordination 
(equipment  failure,  accidents,  foul-ups  of  various  sorts  for  which 

responsibility  cannot  be  fixed) . 
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28  PGirhcrDS  ths  commander'  s  most  difficult/  and  most  important 
task,  is  that  of  developing  an  appropriate  conceptualization  oj. 
the  situation.  By  an  appropriate  conceptualization  we  mean  one 
that  accurately  represents  the  situation  at  the  level (s)  of  detail 
that  is  consistent  with  the  factors  that  a  commander  must  weigh 
in  making  decisions.  It  is  not  necessarily  the  case  that  the  more 
detail  a  model  represents,  the  greater  its  usexulness  will  be. 

3 

29.  A  dimension  in  terms  of  which  C  systems  can  be 
compared  is  the  degree  to  which  decision-making  responsibility  is 
centralized,  as  opposed  to  being  distributed  among  various  system 

components . 

30.  Competition  among  commanders  should  not  be  overlooked  as 
a  factor  that  can  influence  their  performance.  An  interesting 
question  of  some  importance  for  C  systems,  and  for  other  systems 
and  organizations  as  well,  is  the  following  one:  At  what  level  (s) 
within  the  system  should  competition  among  system  units  be  tolerate 
or  even  encouraged?  More  generally,  how  can  compecition  be  used 

to  improve  system  performance  instead  of  being  divisive  and 
counter-productive? 

31.  Is  a  multiprocessor  computer  system  a  useful  analogy 
to  apply  to  the  problem  of  assigning  tasks  among  the  units  of  an 
organization? 


11 
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32.  A  factor  that  needs  to  be  considered  in  the  design  of 
multi-nation  C3  systems  is  that  cf  the  implications  of  specific 
languages  for  thinking.  The  question  is,  to  v;hat  extent  does 
language  constrain  one's  thought  processes,  and  also  what  of 
importance  may  get  lost  in  the  translation  from  one  language  to 
another? 

33.  A  key  problem  of  any  decision-making  situation  is  that 
of  explicating  the  value  space  in  terms  of  which  choices  will  be 
made.  The  problem  is  to  determine  and  make  explicit  the  worth  that 
the  decision  maker  attaches  to  the  various  possible  decision  out¬ 
comes.  This  is  a  nontrivial  problem,  and  one  that  probably  is  not 
solved  simply  by  asking  a  decision  maker  to  assign  utilities  to  a 
set  of  outcome  possibilities. 

34.  Consideration  should  be  given  to  the  possibility  of  using 

3 

a  production— rule  model  to  develop  simulation  mouels  of  C  situahio 
In  particular,  Rand  Corporation's  terrorism  model,  which  is  written 
in  an  "if  ...  then"  formalism  should  be  reviewed. 

35.  The  question  of  how  physiological  factors  affect  problem 
solving  and  decision  making  should  be  investigated,  as  should  the 
question  of  how  cognitive  performance  is  affected  by  mental  and/or 

physical  fatigue. 
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36.  one  way  to  aline  a  goal  in  as  a  desirable  future  world 

state.  An  advantage  to  this  definition  is  that  it  loads 

..  nrociress  resulting  from  goal-directed 

natural  measurement  P 

•  namely  the  amount  by  which  the  distance  between  a 
behavior,  namely, 

.  „ rl<1  state  and  the  desired  world  state  (i.e.,  the  goa 

current  world  stare 

is  thereby  reduced. 

,  r3  c-vstem  as  a  tool  in  the  hands  of  the 

37.  One  may  view  a  C  -i* 

commander .  Alternatively,  one  might  view  the  system  ^  1"C”J 

thB  commander  as  a  component.  Do  these  views  lead  to 

approaches  in  the  study  and/or  operation  of  C  systems? 

38.  The  following  set  of  elements  was  proposed  as  basic 

.  o£  ,my  c3  system:  (a)  goals,  (b>  procedures,  (c)  actron 
components  of  any  y 

,  npnnle-ware)  ,  and  (d)  communications 
elements  (both  hardware  and  people 

channels. 

39.  With  regard  to  the  desirability  of  developing  a 
structural  model  of  C3  systems,  it  was  pointed  out  that  it  is 
important  to  recognise  that  dramatic  changes  in  C3  activities 
may  alter  the  structure  of  a  system  as  well  as  its  modes  of 

operation. 

40.  it  is  important  that  a  commander  be  given  the  right 
amount  of  information--**  much  as  he  needs,  but  not  more  than 
he  can  assimilate.  Given  the  large  amounts  of  information  that 

i  t o  the  situations  in  which  a  high-level 

typically  are  relevant  to  tne 

.  -ir:  -i-t-iq  essential  that  much  of  th( 
commander  might  find  himself,  it  is  essenti 
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information  coming  into  a  system  be  filtered  or  partially 
processed  before  reaching  the  commander.  How  important  is  it 
that  a  commander  understand  where  and  by  whom  the  filtering 
and  processing  are  done,  and  to  understand  the  details,  of  the 
filtering  and  processing  techniques  that  are  used? 


41.  The  importance  of  accurate  aedback  at  various  levels 
in  a  C3  system  was  stressed.  The  absence  of  feedback  was  cited 
as  one  cause  of  system,  or  component,  failure.  The  provision 
of  feedback  that  is  distorted  so  the  commander  will  hear  what 
he  wants  to— or  at  least  not  hear  what  he  does  not  want  to¬ 
wns  suggested  to  be  a  nontrivial  problem. 
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Workshop  #2:  Perception  and  Information  Presentation 

The  successful  operation  of  a  C3  system  depends  on  the 
acquisition,  assimilation,  and  communication  of  many  different 
kinds  of  information,  e.g. ,  sensor  data,  geographical  data, 
written  and  oral  reports  and  commands,  statistical  summaries, 
etc.  At  this  workshop,  we  will  focus  on  problems  relating  to 
information  presentation.  The  general  question  is:  How  should 
various  types  of  information  be  presented  in  order  to  maximize 
the  efficiency  with  which  it  is  perceived  and  used? 


In  order  to  provide  a  stimulus  for  your  thinking  about 
material  that  might  be  relevant  to  our  discussions,  we  have 
listed  some  representative  topics  below,  together  with  a  sampling 
of  questions  that  relate  to  them.  Neither  the  topic  set  nor 
the  questions  should  be  reqarded  as  exhaustive,  and  you  are 
invited  to  make  any  additions  to  either  that  you  feel  are 
appropriate. 


1. 


Written  vs.  Oral  Means  of  Communication 

What  are  the  differences  in  memorability  of  written  and 
spoken  discourse? 


How  much  time  and  effort  does 
in  the  composition  of  oral  vs 


the  sender  tend  to  invest 
written  messages? 


How  does  the  comprehension  of  oral  and  written  messages 
differ  in  terms  of  effort  and  reliability. 


How  does  the  speed,  efficiency  or  coherence  of  message 
composition  differ  between  dictated,  handwritten,  or 
directly  typed  messages? 


Methods  for  Accelerating  Information  Presenta_tion_and 
Assimiliation . 

considering  both  costs  and  intelligibility,  what  is  the 
best  methoc1  for  compressing  speech?  Should  the  cicienon 
algorithm  be  time-based,  or  should  it  concentrate  on 
lels  informative  parts  of  the  signal  lake  long  vowels 
and  pauses?  What  comparable  techniques  could  be  applied 
to  written  information? 
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impor tance  of  Nonlinguistic  JCnformation  in  Spokenjtessages 


How  good  are  people  at  speaker  identification? 


What  are  the  critical  cues  for  speaker  identification? 


What  is  the  prognosis  for  automatic  voice-printing  techniques? 
Can  they  become  a  viable  security  measurer 


How  important  are  prosodic  features  to  the  interpretability 
of  speech? 


How  important  is  the  presence  of  pauses  to  speech  intelli 
gibility  at  the  lexical,  syntactic  and  semantic  levels. 


Atten tion  and  Information-Processing  Capacity 


To  what  extent  can  human  information  processing  capacity 
be  increased  through  multimodal  channels?  Under  wnat 
conditions  is  it  decree sed? 


,  .  .  4-v.«  reliability  of  human  information 

processing  be  increased  through  the  use  of  multimodal  channels? 


Verbal  vs .  Pictorial  Modes  of  Information  Presentation 


What  kinds  of  information  can  be  better  communicated 
through  pictures  than  through  verbal  means? 

How  do  pictures  compare  with  verbal  descriptions  in  terms 
of  speed  of  interpretation? 

What  measures  might  one  use  to  compare  the  interpretability 
or  memorability  of  pictures  vs.  verbal  material? 

Can  reading  efficiency  be  improved  by  embedding  or  sub¬ 
stituting  pictograms  or  pictures  in  prose  material? 

How  does  the  memorability  of  pictures  and  prose  compare? 

What  kinds  of  errors  do  people  make  in  recalling  (verbally 
reconstructing)  pictorial  information  from  memory? 

How  do  different  people's  reconstructions  of  pictorial 
information  compare?  Are  the  distortions  and  omissions 
entirely  idiosyncratic? 

How  does  set  or  preconception  affect  the  encoding  of 
complex  scenes? 

In  map  reading  what  is  the  tradeoff  between  complexity  and 
detail  vs.  interpretability? 
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In  terms  of  coding  variables  what  is  the  best  way  to 
integrate  static  (geographic)  information  with  dynamically 
changing  status  information? 

What  types  of  information  are  best  presented  graphically 
or  pictorially  as  opposed  to  verbally  or  numerical  y. 

Interpretabilitv  of  Various  Hays  of _ D i sp 1 ay ing_Quan t 1 t a t ive 

Information 

What  factors  affect  the  interpretability  of  different  kinds 
of  graphic  representations  of  data? 

What  factors  affect  the  interpretability  of  tabular  data? 

What  kinds  of  information  are  most  naturally  represented 
in  each  of  these  forms? 

what  binds  of  problems  do  people  have  in  recoding  or 
translating  information  from  one  of  these  forms  to  another. 

What  kinds  of  systematic  biases  in  interpretation  are 
associated  with  different  forms  of  statistical  summary? 

How  do  equivalent  representations  differ  in  terms  of 
memorability? 

From  which  kinds  of  summaries  can  people  best  reconstruct 
the  original  geographical  information? 

How  can  complex  interactions  among  variables  be  meaningfully 
represented? 

Perception  and  Visual  Display  Parameters 

Can  the  distinction  between  integral  and  separable  _ 
dimensions  be  exploited  to  enhance  presentation  of  m 
formation? 

What  are  the  relative  advantages  and  disadvantages  of 
large-scale  visual  displays  as  compared  with  small  display 
consoles  positioned  at  individual  user  spaces. 

How  should  displays  of  written  material  be  designed  in 
terms  of  color,  contrast,  resolution,  visual  angle  and 
rate  of  change  for  readability? 

How  should  these  parameters  differ  according  to  the  viewer 
task  (e.g.,  monitoring,  searching,  browsing,  reading  for 

comprehension) ? 

What  should  be  done  in  the  way  of  formatting  visual  layouts 
(e.g.,  space  and  color)  to  maximize  the  viewer  s  ability 
to  organize  and  remember  the  information? 
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8 .  Search  and  Selective  Information  Processing 

What  is  the  feasibility  of  using  eye  fixation  as  an  input 
to  a  computer-driven  display,  e.g.,  to  use  the  point  at 
which  the  user  is  fixating  as  control  information  when  he 
requests  additional  data  regarding  something  on  the  display 


How  does  visual  code  redundancy  affect  visual  search 
performance? 

How  does  familiarity  with  a  spatial  layout  and  location 
of  particular  attributes  on  a  visual  display  affect  the 
time  to  identify  an  attribute  value? 

Is  there  a  difference  in  the  ease  with  which  people  can 
monitor  ("skim")  written  vs.  spoken  material  for  specific 
information? 

How  do  visual  search  processes  differ  when  the  target  is 
physically  vs.  semantically  or  categorically  defined? 

How  do  people  scan  written  material  for  specific  words? 

How  do  people  scan  written  material  for  semantically 
defined  information? 
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Enclosed  are  the  notes  from  the  workshop  held  on  May  24th  and 
Thank  you  again  for  your  contributions. 
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Notes  from  Workshop  No.  2:  Perception  and  Information  Presentation 

The  notes  that  follow  attempt  to  characterize  the  ideas,  concepts 
and  suggestions  that  emerged  from  our  2-day  workshop  involving  . 
individuals  interested  in  basic  and  applied  research  on  perception 
and  information  display.  The  goal  of  this  paper  is  to  present  the 
ideas  introduced  rather  than  to  provide  a  coherent  summary  of  the 
meeting.  It  should  serve  as  a  reminder  to  those  present  of  what  was 
presented  rather  than  as  a  formal  elaboration  of  the  results  of  the 

conference. 

Command,  Control  and  communications  Systems  depend  heavily  on 
the  presentation  and  communication,  of  information.  Commanders  and 
their  staff  have  many  requirements  for  verbal  communications  both  in 
terms  of  speech  and  in  terms  of  formal  written  messages.  Command 
and  Control  Systems  rely  heavily  on  the  presentation  of  data  from 
sensors  in  the  field.  In  some  cases,  these  data  are  in  a  raw  form, 
in  other  cases  they  may  undergo  several  stages  of  processing  and 
refinement  before  they  are  presented  for  use.  Much  of  the 
information  used  by  a  co-ander  and  his  staff  has  a  geographical 
basis.  Many  issues  in  information  display  concern  the  appropriate 
way  to  provide  information  overlays  through  maps  and  charts  that 
indicate  the  relationship  among  elements  of  a  task  force  or  plan. 
Finally  much  information  is  summarized  in  the  form  of  statrs  ics 
graphs,  charts,  and  probabilistic  estimates.  With  this  backgroun 
as  to  the  range  of  potential  applications,  the  group  . 
a  wide  range  of  topics  that  might  be  tentatively  summarized  under 

the  following  headings.  . 

1.  Taxonomy  of  classes  of  information  and  user  requirements. 

2.  information  integration,  perceptualization,  and  global 
feature  analysis. 
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and  mental  chronometry. 


3.  Basic  information  processing 

4.  Psychophysics  of  displays. 

5.  Monitoring,  attention  and  workload. 

6.  Information  overload. 


1.0  TAXONOMY 

The  importance  of  working  towards  a  taxonomy  of  xnformatron 
structures  of  the  sort  required  in  C  systems  was  emphasised.  . 
However,  the  group  was  able  to  contribute  little  of  a  substantive 
sort  about  progress  toward  that  goal.  We  considered  a  taxonomy 
that  begins  from  user  requirements.  We  considered  a  taxonomy  of 
tools  fL  displaying  information  as  a  function  of  the  nature  of 
the  information.  We  considered  the  possibility  of  a  task  analysis 
leading  to  a  taxonomy  and  we  considered  the  case  of  a  taxonomy 
derived  from  elemental  information-processing  operations.  However, 
it  was  pointed  out  that  the  usefulness  of  an  information  taxonomy 

,  .  __  onecifics  of  the  application  both  with 

is  highly  dependent  on  the  speciti 

respect  to  the  level  of  detail  at  which  the  taxonomy  should  be 
developed  and  with  respect  to  the  most  useful  perspective  from 

which  to  derive  it. 


One  possible  taxonomy  of  C 


activities  that  was  proposed  was 


the  following: 


Resource  allocation 

Scheduling 

Monitoring 

Event  classification 
Creation  and  invention 


One  approach  to  the  problem  of  taxonomy  that  was  described 
was  the  formulation  of  a  computer  simulation  of  the  command  and 
control  process  under  study.  Such  a  simulation  would  provide 
for  the  definition  of  the  hierarchical  structure  of  operations 
leading  to  finer  and  finer  analysis  and  perhaps  ultimately  to  the 
description  of  the  elemental  human  information -processing 
operations  required.  Some  skepticism  was  expressed,  however ,  about 
the  practicality  of  carrying  forward  such  a  simulation  for 
situations  as  open-ended  and  unstructured  as  the  typical  command 
and  control  application. 
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Dimensions  of  C  system  requirements  that  might  have  usefulness 
for  formulating  taxonomies  included: 

1.  The  extent  to  which  possible  decisions  to  be  made  can 
be  specified  in  advance  and  anticipated. 

2.  The  extent  to  which  time  stress  is  a  critical  variable 
in  performance. 

3.  The  requirement  for  analog  versus  digital  data  abstraction. 

2.0  INFORMATION  INTEGRATION 

The  underlying  concept  that  was  the  focus  of  this  discussion 
of  information  integration  was  the  idea  that  information  should  be 
presented  in  such  a  way  as  to  make  it  possible  to  incorporate  it 
into  the  use  's  model  of  the  object  or  objects  under  surveillance. 
Modes  of  presentation  should  enhance  the  coherence  among  the 
dimensions  of  data  in  order  to  relate  them  to  the  underlying  model. 
It  was  suggested  by  several  individuals  that  the  archtype  of  the 
expert  is  his  ability  to  conceptualize  his  activities  in  a  model 
of  the  process  with  which  he  is  working  that  makes  it  easy  to 
incorporate  new  data  or  information  into  the  model.  It  is  as  if 
the  expert  is  able  to  detect  and  operate  on  more  complex  patterns 
at  one  time  than  the  novice. 


Sometimes  it  is  desirable  to  alter  the  time  scale  over  which 
information  is  presented  in  order  to  enhance  its  coherence  or 
perceptability.  This  led  to  a  discussion  of  the  possibilities 
for  speeding  up  speech  signals  and  to  the  question  of  the  condi¬ 
tions  under  which  speeded  speech  might  be  useful  in  a  C  system. 
The  following  purposes  were  offered  as  possibilities. 

•1.  A  long  message  might  be  monitored  for  target  words  in 
order  to  find  a  part  of  a  message  to  be  attended  to  in 
more  detail. 

2.  Speeded  speech  might  be  used  for  review  of  a  briefing 
session  where  it  is  desired  to  browse  through  the  speech 

signal . 

3.  Sometimes  visual  information  that  is  synchronized  with 
speech  needs  to  be  presented  at  a  rate  faster  than  real 
time.  Under  these  conditions  it  might  be  useful  to 
speed  up  the  speech  accordingly  in  order  to  pace  it  to 
the  time  scale  in  which  the  visual  information  is  to  be 
presented . 

It  was  agreed  that  if  the  chopping  algorithm  is  correctly 
chosen  speech  can  be  speeded  up  by  a  factor  of  approximately 
3  to  1  and  maintain  intelligibility. 

Returning  to  the  more  general  subject  of  speeded-up  visual 
or  auditory  signals,  the  question  was  raised  concerning  ways  to 
pick  the  optimum  integration  window  or  the  optimum  presentation 
rate.  It  was  suggested  that  either  control  of  the  rate  can  be 
given  to  the  user  to  adapt  to  his  own  needs  or  that  it  might  be 
possible  to  define  a  formal  basis  for  choosing  an  integration 
window  by  considering  the  rate  of  change  of  the  phenomenon  under 
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study  in  relation  to  the  rate  of  change  of  the  noise  in  which  the 
phenomenon  is  imbedded.  It  was  also  suggested  that  at  times  there 
might  be  value  in  slowing  down  signals  as  well  as  speeding  them  up. 
The  purpose  here  would  be  to  defccus  attention  on  the  coherent 
factors  to  get  at  other  content  that  may  not  be  evident  because 
of  the  compelling  nature  of  the  coherent  events.  With  respect  to 
speech  this  was  suggested  as  a  possible  way  to  focus  on  the 
emotional  content  of  the  speech. 

Other  techniques  for  enhancing  the  perceptibility  of  coherent 
signals  were  discussed.  At  times  it  may  be  useful  to  transform  a 
signal  represented  in  the  time  domain  into  one  represented  in  the 
spatial  domain.  The  common  example  is  the  use  of  a  sound  spectrogram 
A  further  suggestion  concerned  crossmodal  integration.  A  particular 
selected  feature  of  the  visual  display  may  be  a  momentarily 
relevant  source  of  information.  As  an  attention-getting  device 
it  might  be  appropriate  to  modulate  the  intensity  of  that  visual 
feature  with  a  voice  signal  presented  simultaneously  so  that  the 
voice  modulation  and  the  fluctuation  of  the  visual  pattern  would 
be  coherent  with  each  other. 

A  topic  of  some  substantial  interest  was  the  discussion  of 
the  use  of  facial  features  to  represent  multi-dimensional  data. 
Chernoff  in  two  papers  in  the  Journal  of  the  American  Statistical 
Association  (1973)  has  shown  that  if  the  attributes  of  multi¬ 
dimensional  stimuli  are  represented  in  terms  of  changing  features 
of  a  set  of  stereotyped  faces,  the  display  of  multiple  arrays 
of  such  faces  enhances  the  observers  ability  to  detect  clustering 
and  patterning  in  the  data.  It  was  suggested  that  the  relative 
simplicity  of  visual  inspection  of  these  facial  arrays  can  work 
almost  as  successfully  as  formal  computational  algorithms  for 
computing  many  potential  interactions  and  that  the  formal 
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computing  algorithm  is  very  expensive  and  time  consuming  even 
with  today's  high-speed  computers. 

Howard  Egeth  (Jacob,  Egeth  and  Bevan,  1976)  and  V7illiam  Huggins 
at  John  Hopkins  University  have  continued  work  with  these  facial 
representations.  Egeth  has  used  a  classification  paradigm  and  has 
shown  that  classification  of  data  represented  in  faces  is  more 
efficient  than  representation  of  the  same  data  in  matrices  or  in 
terms  of  multidimensional  abstract  symbols.  There  appears  to  be 
something  unique  about  the  coherence  of  the  faces.  He  pointed  out 
that  while  accuracy  is  higher  with  faces  than  with  other  data  display 
classification  performance  is  still  not  sufficiently  reliable  to  make 
it  a  particularly  useful  technique  in  a  practical  setting. 

3.0  INFORMATION  PROCESSING  AND  MENTAL  CHRONOMETRY 

The  discussion  of  mental  chronometry  opened  with  Bill  Chase's 
presentation  of  four  circumstances  in  which  attempts  to  develop 
predictions  of  the  time  required  for  task  performance  depended 
primarily  on  the  nature  of  the  data  base  with  which  the  subjects 
were  operating  and  on  the  ways  in  which  they  organized  themselves 
to  make  use  of  that  database,  rather  than  on  time  required  for  the 
elemental  information  processing  operations  that  were  performed 
per  se.  The  tasks  included  the  analysis  of  skilled  chess  players, 
the  analysis  of  a  text-editing  problem,  and  the  analysis  of  the 
calculating  performance  of  mental  wizards.  In  each  case,  it  was 
argued  that  the  efficiency  of  the  mental  processing  depended 
largely  on  the  way  in  which  they  organized  the  job  and  only 
indirectly  on  the  nature  of  the  information  processing  operations. 


Thus  it  was  argued  that  mental  chronometry  is  not  particularly 
suited  for  predicting  molar  behavior  f  rom  analysis  of  its  micro¬ 
structure,  but  it  was  agreed  that  it  does  provide  a  way  of  thinking 
about  information  processing  activities  and  can  tell  us  something 
about  the  way  in  which  information  processing  elements  or 
structures  are  integrated  and  used.  It  is  more  of  a  diagnostic 
tool  than  a  predictive  one. 


The  question  was  then  raised  concerning  the  conditions  under 
which  pictures  arc  a  preferred  mode  of  display  as  opposed  to  words 
or  verbal  material.  it  was  argued  that  patterns  may  be  more 
easily  related  to  distinctive  response  modes,  and  therefore  if 
compatibility  between  the  visual  input  and  manual  output  is  importat 
then  pictures  may  be  preferred.  It  was  suggested  that  spatially- 
distributed  information  is  probably  better  presented  in  pictures 
while  information  that  has  a  temporal  distribution  is  better 
presented  in  printed  words  or  in  a  speech  signal.  Words  are 
believed  to  be  useful  for  reducing  biases  in  perception,  and,  of 
course,  they  are  needed  to  talk  about  concepts  for  which  corres¬ 
ponding  pictorial  representations  are  not  available .  In  a  reaction  ti 
paradigm  it  takes  appioximately  250  milliseconds  longer  to  name  a 
picture  than  to  name  a  word  and  this  fact  should  be  taken  into 
account.  Of  course,  it  may  be  misleading  because  additional  proce-s 
ang  time  may  have  to  be  spent  in  finding  the  word  to  name  in  order"’ 

to  consider  the  conditions  equivalent  to  those  under  which  pictures 
are  named. 


This  led  to  a  discussion  of  Molly  Potter's  work  on  the 
integration  of  words  and  pictures  into  sentences.  She  has 
found  that  under  tachstoscopic  presentation  conditions, 
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the  selected  substitution  o£  pictures  for  words  does  not  substan¬ 
tially  reduce  the  interpretability  of  a  sentence.  It  seems  clear, 
however ,  that  this  capability  depends  heavily  on  the  redundancy 
and  predictability  of  the  sentence  content.  It  might  also  be 
noted  that  the  same  principle  probably  does  not  apply  at  the 
phonemic  level,  as  evidenced  by  the  difficulty  of  interpreting 

this  expression:  ' — eraic' 


The  question  of  map  reading  and  interpretability  were  the  . 
natural  outgrowth  of  the  discussion  of  pictures  and' words,  although 
it  occurred  at  a  later  time  in  the  workshop.  It  was  repeatedly 
emphasized  that  maps  and  geography  form  the  basis  for  many  of  the 
kir 1o  of  displays  of  interest  to  a  commander  and  his  staff  in  a 
C3  system.  An  interesting  and  potentially  rosearchable  question 
centered  on  how  to  design  maps  to  aid  human  inference  processes 
with  respect  to  them.  Maps  are  inherently  hierarchical  in 
organization  and  it  was  argued  that  inferences  about  maps  may  also 
be  so  organized.  At  difforent  times  the  user  requires  different 
features  of  a  map  to  be  emphasized.  It  was  argued  that  a  map  can 
form  a  kind  of  schema  for  organization  of  data. 


While  geography  may  form  the  basis  for  the  schematic  repre¬ 
sentation,  the  geographic  orientation  may  be  distorted  in  order  to 
enhance  the  interpretability.  Figures  1  and  2  are  submitted  as  an 
illustration  of  this  point.  They  show  two  representations  of  tie 
ARFA  computer  network.  One  lays  out  the  sites  superimposed  on  tho 
geography  of  tho  United  States.  The  second  shows  a  logical  repre¬ 
sentation  of  the  sites  with  considerably  more  detail  concerning  the 
oharacteristies  of  the  individual  sites.  Both  are  obviously  useful 
representations  for  different  purposes. 
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(PLEASE  NOTE  THAT  WHILE  THIS  MAP  SHOWS  THE  HOST  POPULATION  OF  7  HE  \ETWORK  ACCORDING  TO  THE  BEST 
INFORMATION  OBTAINABLE,  NO  CLAIM  CAN  BE  MADE  FOR  ITS  ACCURACY  > 


A  novel  concept  for  the  presentation  of  geographical  informa¬ 
tion  at  different  levels  of  detail  makes  use  of  an  extension  of 
the  concept  of  zooming.  If  the  information  in  a  geographical 
display  is  represented  in  the  computer  hierarchically,  it  is 
possible  to  zoom  to  different  levels  in  the  hierarchy.  These 
levels  may  correspond  to  levels  of  magnification  but  they  may  also 
correspond  to  different  levels  of  logical  detail.  Thus  cne  might 
move  from  a  map  of  the  United  States  to  a  map  of  a  particular 
ARPANET  site.  Further  zooming  would  then  present  more  detailed 
information  about  the  characteristics  of  that  particular  site, 
perhaps  even  characteristics  of  the  computer  located  at  that  site. 

The  user  would  need  a  technique,  such  as  a  pointer,  to  designate  the 
focus  of  the  expanded  scale  information,  as  well  as  to  be  able  to 
designate  the  nature  of  the  more  detailed  information  he  desired. 

In  order  to  maintain  orientation  with  respect  to  a  geographical 
display  some  kind  of  reference  points  are  required.  A  question  was 
raised  concerning  whether  the  reference  points  should  be  static  or 
whether  they  could  be  defined  relative  to  the  last  previous 
presentation.  It  was  argued  that  for  some  purposes  the  long  term 
conceptualization  of  the  geography  and  fixed  referenced  points  are 
important  while  for  other  purposes  one  may  be  only  interested  in 
information  relative  to  the  last  point  examined. 

Bill  Chase  pointed  out  that  a  great  deal  of  expertise  exists 
in  the  National  Geographic  cartographic  group  particularly  concerning 
the  use  of  colors,  the  print  characteristics,  and  so  forth.  As  far 
as  the  level  of  details  to  be  presented  was  concerned  it  was 
asserted  that  the  cartographic  service  seems  to  make  use  of  a 
criterion  of  the  number  of  details  presented  per  unit  area.  In 
areas  where  the  density  is  great,  they  present  fewer  details  and  in 
the  area  where  the  density  is  thin  they  tend  to  present  more  details. 
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The  issue  of  defining  fixed  reference  points  led  to  the 
discussion  of  whether  maps  should  be  viewed  as  inside-out  or 
outside-in  displays.  Taking  the  outside-in  view  suggests  that 
maps  should  always  be  oriented  with  north  to  the  top  with  fixed 
landmarks  always  at  the  same  relative  position.  If  one  takes  an 
inside-out  view,  then  the  map  should  be  oriented  so  that  the  user 
can  associate  the  direction  of  movement  with  realistic  assessments 
of  left  and  right.  The  inside-out  view  seems  most  appropriate 
when  maps  are  used  for  navigational  purposes,  however,  in  command 
and  control  applications,  maps  are  seldom  used  in  that  sense. 

There  remains,  however,  the  question  of  whether  or  not  the  user's 
location  on  the  map  should  serve  as  the  reference  point  and  the 
map  oriented  with  the  field  of  battle  or  whatever  arrayed  about 
that  point  as  the  central  focus.  This  seems  like  another  case  for 
which  the  answer  can  only  be  given  in  an  application-specific  case. 

With  respect  to  maps,  the  final  question  considered  was  that 
of  suitable  means  for  data  entry  into  a  map  display.  When  one 
needs  to  enter  new  data  or  update  old  data  it  becomes  necessary  to 
articulate  a  means  for  getting  those  new  data  to  the  map  in  the 
proper  location.  While  a  lightpen  or  pointing  device  is  suitable 
for  locating  positions  to  enter  the  data,  digital  data  are  most 
easily  entered  on  a  keyboard.  Sometimes  coordination  of  a  lightpen 
and  keyboard  are  awkward.  It  is  also  necessary,  not  only  to 
identify  the  location  on  the  map  to  which  the  data  refer,  but  also 
to  designate  their  location  in  the  supporting  database.  If  these 
designations  are  to  be  made  by  pointing,  then  the  place  where  the 
data  are  to  be  entered  must  be  coded  with  respect  to  their  database 
category.  If,  on  the  other  hand,  the  database  category  is  entered 
by  keyboard,  this  usually  results  in  a  requirement  for  a  multi¬ 
dimensional  argument  to  designate  the  specific  location  in  which  it 
is  to  be  entered.  Thus  there  are  a  number  of  combinations  of 
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pointing  and  keyboard  data  entry  techniques  that  are  useful  for 
data  entry  in  a  graphical  display.  However,  this  seems  like  a 
topic  for  which  further  research  on  the  relative  effectiveness  of 
alternative  data  entry  techniques  would  be  productive. 

4.0  THE  PSYCHOPHYSICS  OF  DISPLAYS 

Harry  Snyder  gave  a  presentation  concerning  the  current 
status  of  efforts  to  develop  quantitative  methods  for  evaluating 
display  quality.  His  group  at  VPI  has  been  working  on  the 
application  of  the  sinusoidal  modulation  transfer  function  for  this 
purpose.  Most  previous  display  evaluation  methods  have  focused 
on  the  maximum  spatial  frequency  at  which  bars  or  sinusoidal 
modulation  may  just  be  detected.  However,  as  Harry  points  out, 
this  kind  of  measure  focuses  on  a  single  point  in  a  space  defined 
by  the  functional  relation  between  optical  response  and  modulation 
frequency.  There  is  a  large  supra-threshhold  region  in  which  the 
optical  response  characteristics  have  a  significant  impact  on 
overall  display  quality.  Harry's  group  has  been  working  with  a 
measure  that  corresponds  to  the  area  under  the  curve  relating 
optical  response  to  sinusoidal  modulation  frequency  that  lies  above 
the  visual  detection  threshhold.  He  demonstrated  empirically  that 
this  measure  predicts  suqcess  at  several  symbol  and  target 
identification  tasks  by  human  observers. 

His  group  has  also  been  working  with  the  development  of 
optimal  display  fonts  for  dot  matrix  characters.  Previous  work 
has  focused  on  the  development  of  display  fonts  for  stroke 
characters.  Work  was  described  suggesting  that,  if  the  constraints 
of  a  5  x  7  matrix  are  imposed,  a  font  of  their  own  design  produces 
significantly  better  legibility  than  the  Lincoln-Mitre  font  which 
was  originally  designed  for  stroke  characters. 


Dot  matrix  characters  also  suffer  from  the  interference  of 
the  raster  scan  lines  in  a  raster  scan  display.  Harry  presented 
data  suggesting  that  the  legibility  may  be  improved  if  a  small 
amount  of  jitter  is  introduced  in  the  position  of  the  dots.  This 
has  the  effect  of  defocusing  the  raster  lines  and  softening  the 
overall  display  thereby  improving  legibility. 

The  use  of  colors  for  display  enhancement  was  also  discussed. 

For  categorical  search  tasks  color  has  been  shown  to  be  better 
than  some  other  coding  dimensions  for  enhancing  the  speed  and  accurac 
in  such  a  search.  Christ  (1975)  has  reviewed  the  data  on  color 
as  a  coding  dimension  in  a  1975  Human  Factors  s  article.  Snyder 
also  asserted  that  the  use  of  pseudocolor  to  enhance  contrast 
turns  out  to  be  less  effective  than  to  directly  manipulate  the 
contrast  range  on  a  black  and  white  image.  Snyder  mentioned  that 
in  terms  of  its  modulation  transfer  function,  color  does  not  have 
as  broad  a  response  band  as  direct  gray-scale  illumination  does. 
Thus,  purely  from  a  psychophysical  point  of  view,  black  and  white 
would  be  preferred  to  a  single  color  display.  Finally,  it  was 
pointed  out  that  the  choice  of  colors  as  coding  dimensions  should  be 

selected  consistent  with  the  nature  of  the  semantic  features  to  be 
represented.  If  a  temperature  is  being  represented,  it  should  be 
in  shades  of  orange-red,  if  depth  of  the  ocean  is  being  represented 
it  should  be  darkness  of  blues,  and  so  forth. 


5.0  MONITORING,  ATTENTION  AND  WORKLOAD 

It  seemed  clear  to  the  participants  that  there  are  very  few 

jobs  in  a  system  that  have  the  classical  properties  of  the 

vigilance  paradigm.  Few  individuals  monitor  radar  scopes  today. 

Further,  it  was  asserted  that  many  of  the  jobs  involved  in  command 

and  control  systems  do  not  involve  exposure  to  time  pressures. 

They  are  more  like  long-range  planning  and  problem  solving 

activities  without  a  moment— to-moment  time  constraint.  However, 

some  activities  might  be  classified  as  supervisory  monitoring, 

a  job  akin  to  that  of  the  power-plant  control-room  supervisor. 

Most  of  the  time  there  is  little  to  do  but  when  the  infrequent 

critical  events  occur,  the  supervisor  needs  to  have  full 

understanding  of  the  current  status  of  the  system.  Job  design  to 

make  this  continually  updated  status  easy  to  accomplish  is  a  major 

3  .  . 

challenge.  A  further  challenge  to  the  C  system  designer  is  to 
create  conditions  that  promote  the  maintenance  of  operator  pro¬ 
ficiency  for  the  many  different  kinds  of  activities  that  he  might 
be  called  upon  to  perform  but  which  occur  very  infrequently.  Such 
systems  make  much  use  of  exercises  for  this  purpose  and  it  was 
argued  that  off-line  simulation  is  another  effective  tool  for 
maintaining  proficiency. 

The  trend  in  designing  supervisory  monitoring  tasks  is  to 
provide  alarms  or  annunciators  when  a  particular  state-variable 
exceeds  a  preset  tolerance.  The  difficulty  is  that  out-of-toleranc 
conditions  arc  frequently  signalled  by  patterns  of  changes  in  state 
variables  rather  than  in  the  behavior  of  individual  state  variables 
It  is  very  difficult  to  design  alarms  that  are  sensitive  to  inter¬ 
actions  among  state  variables,  especially  when  these  interactions 
are  difficult  to  anticipate  for  the  range  of  possible  conditions 
that  may  be  encountered.  Here  again  the  problem  is  to  create  a 


display  system  that  enhances  the  capability  for  visualizing  the 
interrelations  among  task  elements,  a  common  theme  through  much 
of  our  discussion  of  visual  information  presentation. 

6.0  THE  PROBLEM  OF  INFORMATION  OVERLOAD 

Although  not  discussed  as  a  separate  topic,  the  problem  of 
dealing  with  overly  rich  multi-dimensional  sources  of  information 
was  a  theme  that  materialized  at  several  points  during  the 
two-day  workshop.  It  was  asserted  that  a  commander  has  a  tendency 
to  focus  on  only  a  subset  of  the  information  available  to  him  and 
to  formulate  hypotheses  based  on  that  subset  of  information. 
Frequently  he  will  neglect  important  information  that  would 
influence  his  choice  of  hypotheses.  The  problem  is  exaggerated 
because  once  having  seized  upon  a  particular  hypothesis  he  becomes 
committed  to  it  and  it  takes  an  inordinate  amount  of  disconf irming 
data  to  change  the  commander's  opinion.  The  design  problem  is  to 
assure  that  the  commander  is  presented  with  the  information  that 
is  most  relevant  to  the  issue  he  is  addressing.  But  this  in  turn 
implies  fore— knowledge  about  what  will  be  important.  Techniques 
designed  to  preserve  the  commander1 s  options  before  a  commitment 
is  made  would  be  very  helpful  at  this  stage  of  information 
management. 

If  one  asks  a  commander  "What  information  he  needs  for  a 
particular  operation,"  he  is  likely  to  say  he  needs  to  have 
potential  access  to  everything.  However,  the  ultimate  in  flexibility 
requires  inordinately  complex  retrieval  techniques  and  it  seems 
likely  that  a  point  of  diminishing  returns  will  be  reached.  There 
is  a  tradeoff  between  the  amount  of  information  potentially 
available  to  the  commander  and  the  time  that  will  be  required  to 
access  it.  While  good  information  system  design  may  reduce  the 
severity  of  this  trade-off  it  cannot  eliminate  it.  This  issue  not 


16 


only  involves  the  design  of  displays  and  display  format,  but 
also  goes  to  the  heart  of  the  question  of  information  retrieval 

from  a  database. 

7 . 0  MISCELLANEOUS 

A  number  of  questions  were  raised  which  are  not  easily  subsumed 
under  the  headings  identified  above.  They  will  be  listed  here. 

1.  Are  there  specific  display  design  considerations  that 
can  reduce  the  difficulty  of  working  under  stressful 
conditions?  The  traditional  argument — that  design  to 
take  account  of  stimulus-response  compatibility  and 
good  human  factors  practice  in  general  is  also  likely  to  be 
a  good  design  for  stress  resistance--was  mentioned.  It 
was  also  suggested  that  a  good  design  for  stress  conditions, 
particularly  task-induced  stress,  might  involve  prioritizing 
the  subtasks  in  advance  so  that  under  time  pressure  those 
of  lesser  importance  may  be  left  out.  If  tasks  could  be 
prioritized  then  the  workload  could  be  reduced  by  design 
rather  than  by  fiat.  A  further  suggestion  considered  the 
possibility  of  identifying  particular  tasks  that  are 
inherently  disruptab.le  under  stress.  It  is  these  tasks 
to  which  particular  design  efforts  should  be  put  and  the 
system  made  forgiving  of  disruptions  of  them.  These  might 
be  the  first  tasks  for  which  the  possibility  of  automation 
would  be  considered. 

2.  A  researchable  issue  concerns  the  development  of  content- 
driven  communications.  It  seems  likely  that  artificial 
intelligence  concepts  could  be  applied  to  selectively 
channel  communications  to  the  individual  who  is  concerned 
with  particular  content.  If  we  developed  a  profile  of 


interests  of  a  commander  and  analyzed  the  content  of 
messages  to  look  for  material  related  to  that  profile 
we  would  reduce  the  requirement  for  pre-screening  messages 
and  provide  a  basis  in  advance  for  finding  the  information 
that  is  potentially  important  to  a  particular  commander. 

3.  A  method  for  identifying  the  constraints  on  current  C 

systems  was  suggested  based  on  an  analogy  with  a  study 

of  Paul  Rosen  on  teaching  children  to  read.  Rosen 

analyzed  the  evolutionary  stages  that  led  to  modern 

language  from  the  beginning  of  time.  He  then  formulated 

tests  of  the  capabilities  required  of  readers  at  each 

stage  beginning  with  elementary  pictographs  and  hieroglyphs 

and  proceeding  through  phonic  and  symbolic  presentations 

to  the  current  predominant  use  of  an  abstract  alphabet. 

He  administered  the  tests  to  a  set  of  good  and  poor 

readers  and  found  that  both  groups  performed  equally  well 

on  the  tests  until  they  reached  the  point  of  using  an 

arbitrary  alphabet.  The  analogy  suggests  conducting  a 

3 

study  of  the  evolutionary  development  of  C  systems 

having  as  its  goal  to  identify  a  point  in  history  at  which 

major  breakdowns  in  effectiveness  began  to  occur  and  to 

try  to  pinpoint  those  breakdowns  with  a  particular  point 

in  the  development  of  the  technology  used  to  support  such 

systems.  By  this  means  we  might  identify  weak  links  in 
3 

C  performance  and  thereby  find  the  point  at  which  to  focus 
the  resources  for  system  improvement. 
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4.  In  the  current  international  climate  the  question  was  raised 
as  to  whether  there  were  design  considerations  that  would 

tend  to  eliminate  the  potential  for  deception  and  counter 

3 

measures.  It  was  suggested  that  perhaps  the  optimal  C 
system  today  is  one  that  makes  it  clear  to  the  enemy  what 
information  we  have  and  what  our  intentions  are  in  a  way  that 
precludes  the  suspicion  of  deception.  Is  there  a  technological 
means  to  insure  trust  in  the  output  of  a  system?  Is  there  a 
way  to  provide  group  access  to  a  database  of  information  that 
we  want  all  parties  to  be  aware  of  and  believe  in?  While 
clearly  we  would  not  want  to  expose  all  information  in  such 
a  way,  the  ability  to  expose  it  in  a  way  that  could  be 
guaranteed  to  be  believed  might  be  a  signficant  step 
toward  avoiding  major  overreactions. 


R.  W.  Pew 
9  July  1976 
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Questions  for  C3  Meeting  on  Memory  and  Data  Storage 

lm  what  techniques  are  available  for  data  storage  and  retrieval 
that  would  be  particularly  useful  for  C3  applications? 

2.  Are  any  of  the  methods  that  are  used  by  AI  researchers 
for  representing  knowledge  in  a  computer  memory  sufficiently 
developed  to  be  practically  applicable? 

3.  What  sorts  of  memory  aids  would  be  useful  for  commanders 
and  other  users  of  C3  systems? 

4.  What  kinds  of  browsing  techniques  could  be  made  available 
to  permit  a  user  of  a  C3  system  to  browse  through  a  data  base 
or  some  specified  portion  thereof? 

5.  What  types  of  problems  and  difficulties  encountered  in  C3 
operations  are  most  likely  to  be  attributable  to  human  memory 

limitations? 

6.  How  important  is  it  for  a  user  of  a  computer  data  base  to  have 
a  clear  conceptual  model  of  the  way  the  data  in  that  data  base 
are  organized? 
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7  what  sorts  o£  techniques  can  be  provided  to  the  user  of  a  very 
large  data  base  to  permit  him  to  create  small  data  structures  that 
are  specially  tailored  to  his  own  needs? 


memorandum 


To: 

From: 

Subject: 


Participants  in  Workshop  on 
Memory  and  Information  Storage 

R.  S.  Nickerson 
Notes  from  meeting 


Date: 


10  December  1976 


Enclosed  is  a  set  of  notes  from  the  workshop  on 
Memory  and  Information  Storage  in  C  Systems  that  was 
held  at  BBN  on  28,29  June,  1976.  We  have  made  no 
to  produce  "minutes"  of  the  meeting,  or  even  to  organrze 
the  notes  in  any  particular  way.  We  simply  jotted  down 
the  points  that  were  made,  more  or  less  in  the  order 
in  which  they  were  made  at  the  meeting .  Some  o  em 
will  undoubtedly  prove  to  be  a  bit  cryptic;  however, 
they  may  serve  to  complement  or  supplement  your  own  notes. 

!  want  to  thank  you  for  attending  the  workshop  and 
for  your  contribution  to  the  discussions.  I  hope  you 
found  it  of  some  value. 


Bolt  Beranek  and  Newman  Inc. 


NOTES  FROM  27,28  JUNE,  1976  MEETING  RE  MEMORY  AND  INFORMATION  RETRIEVAL 

}  J 

1.  There  are  at  least  two  ways  in  which  research  on  displays 
and  information  presentation  relates  to  the  question  of  memory 
and  information  storage. 

a.  The  displays  may  function  in  some  cases  as  extensions 
of  memory.  A  formatted  display  that  is  used  in  conjunction 
with  the  user's  input  to  a  computer  system,  for  example, 
may  make  it  unnecessary  for  the  user  to  remember  all  the 
details  of  the  input  by  prompting  him  concerning  the 
information  that  he  must  supply.  Use  of  map  displays  also 
illustrates  the  point.  They  extend  the  user’s  general 
memory  for  geographical  relationships  and  make  it  unnecessary 
for  him  to  remember  all  the  details. 


b.  How  information  is  organized  on  a  display  may 
determine  to  some  degree  how  well  the  display's  material 
will  be  remembered.  In  general,  displays  that  are  consistent 
with  the  familiar  Gestalt  rules  of  organization  will 
probably  be  remembered  better  than  those  that  do  not  obey 
these  rules.  The  determination  of  specific  way"  in  which 
display  organization  affects  memory  performance  is  an  area 
that  could  profit  from  more  research. 


Bolt  Beranek  and  Newman  Inc. 


2.  There  is  a  trade-off  between  structure  of  information 
in  memory  and  inference  capabilities.  Presumably,  the  more 
highly  structured  information  is  in  memory,  the  more  readily 
accessible  it  will  be,  and  therefore  the  less  the  need  for 
inferencing  capabilities. 

3.  Two  assumptions  are  standardly  made  concerning 
memory : 

a.  That  there  is  no  forgetting  in  long-term  memory,  and 

b.  That  memory  is  composed  of  stable  interconnected 

structures . 

4.  There  is  a  problem  of  contradictory  information  in  memory. 
Suppose,  for  example,  that  one  stores  an  hypothesis  that  is  to  be 
tested  by  incoming  data,  and  the  data  prove  the  hypothesis  to  be 
false.  If  the  initial  hypothesis  cannot  be  forgotten,  then  the 
fact  that  it  is  false  must  also  be  stored.  The  result  is  that  over 
time  much  false  information  accumulates  in  memory,  and  this  creates 
a  problem  for  memory-dependent  inferential  processes. 

5.  An  ubiquitous  problem  in  the  study  of  memory  is  that 
of  distinguishing  between  things  that  are  truly  remembered  and 
those  that  are  constructed,  or  inferred.  It  is  not  safe  to  assume 
that  one  can  tell  the  difference  with  respect  to  the  things  that 
one  thinks  are  in  one's  own  memory. 
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6.  A  closely  related  problem  stems  from  the  fact  that 
the  act  of  recall  may  change  a  memory  trace.  This  seems 
especially  likely  to  be  true  when  one  has  to  work  to  recall 
something  in  the  first  place.  The  question  is,  when  something 
is  recalled  for  the  nth  time,  how  is  the  content  of  that 
recall  affected  by  the  fact  that  the  "same  thing"  has  been 
recalled  n-1  times  before? 

7.  An  applied  problem  that  was  described  is  that  of 
determining  how  best  to  encode  communications  between  air  traffic 
controllers  and  pilots,  so  as  to  minimize  memory  loss. 

8.  There  are  two  general  problems  associated  with  the 
design  of  displays  that  are  intended  to  display  information 
that  has  to  be  remembered:  (a)  designing  for  optimal  discrimin- 
ability  or  interpretability ,  and  (b)  designing  for  optimal 
memorability.  A  question  of  practical  significance  is  whether 
designs  that  meet  one  of  these  objectives  also  meet  the  other. 

9.  The  point  was  made  that  optimality  should  be  defined 
not  only  in  terms  of  the  probability  of  error  but  in  terms  of 
the  types  (magnitudes)  of  errors  that  are  made. 

10.  The  effects  of  coding  strategies  on  short-  versus 
long-term  retention  was  mentioned  as  a  general  research  topic. 
Presumably,  some  encoding  strategies  may  work  well  for  short¬ 
term  retention,  but  not  for  long-term  retention,  and  others  may 

do  the  reverse. 
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11.  In  some  cases  it  is  important  to  forget  information 
after  it  has  been  used.  For  such  cases,  one  wants  encoding 
strategies  that  will  facilitate  the  long-term  forgetting. 

12.  A  distinction  was  made  between  remembering  when  one 
knows  the  purpose  for  (use  of)  the  information  that  is  being 
retained  and  the  case  in  which  the  use  of  that  information  is 
not  known  in  advance.  Conjecture:  knowledge  of  how  information 
is  to  be  used  might  facilitate  the  encoding  of  that  information 

to  maximize  its  retention  and  accessibility  for  that  particular  use 

13.  There  may  be  a  difference  between  remembering  facts 
and  remembering  how  to  retrieve  facts. 

14.  The  concept  of  "memory  pollution"  was  introduced.  The 
idea  is  that  the  retention  and  retrieval  of  important  facts  may 
be  impaired  if  memory  is  clogged  up  with  a  great  deal  of  trivia. 

The  notion  seems  to  necessitate  one  or  two  assumptions:  (a)  that 

memory  has  relatively  small  capacity  so  that  the  storage  of  some 
elements  may  preclude  the  storage  of  others,  or  (b)  that  there 

is  a  great  deal  of  interference  among  the  stored  items  of  informa¬ 
tion. 

15.  One  of  the  difficulties  in  studying  human  memory  is 
that  of  distinguishing  between  what  is  remembered  and  what  is 
produced,  i.e.,  inferred  at  the  time  of  recall.  People  are 
motivated  to  be  consistent  and  perhaps  complete  in  what  they 
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recall,  and  therefore  may  be  inclined  to  interpolate, 
extrapolate,  and  in  other  ways  generate  information  that 
they  do  not  explicitly  remember.  Moreover,  it  is  not  safe 
to  assume  that  people  can  themselves  distinguish  when  they 
are  recalling  something  from  when  they  are  generating  it. 

16.  What  people  "remember"  from  events  they  have 
witnessed  may  depend  very  much  on  the  way  in  which  they  are 
questioned  about  those  events.  A  skillful  questioner 
apparently  may  be  able  to  lead  an  individual  to  believe  that 
he  remembers  something  that  he  in  fact  did  not  witness. 

17.  It  was  suggested  that  if  one  wants  to  do  research 

on  human  memory  (or  other  memory  processes)  that  will  be 

3 

relevant  to  the  operation  of  C  systems,  one  might  begin 
by  attempting  to  identify  the  various  types  of  memory  path¬ 
ologies  that  are  found  in  these  systems.  Among  the  pathologies 
that  one  might  expect  to  find  are  the  following: 

a.  Overconfidence  in  one's  memory  and  in  the  ability 
to  retrieve  needed  information  from  it 

b.  Difficulties  associated  with  updating  of  long-term 
memory  (either  human  or  machine) 

c.  Poor  organization  of  stored  information 

d.  Insufficiently  effective  mnemonic  aids  for  getting 
information  out  of  memory 
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18.  The  structure  and  sequencing  of  material  during 
presentation  are  important  determinants  of  retention.  In 
part,  because  of  this  fact,  the  problem  of  updating  memory 
is  difficult  because  one  must  be  concerned  about  how  the 
new  information  fits  within  the  structure  that  has  been 
imposed  on  the  old  information. 

19.  Relatively  little  is  yet  known  about  mnemonic 
codes  and  their  effectiveness.  This  was  suggested  as  a 
fruitful  area  for  research. 

20.  The  need  to  develop  ways  to  study  memory  as  it 
"naturally"  functions  was  stressed.  The  generalizability 

of  the  results  obtained  in  many  laboratory  studies  of  memory 
is  highly  questionable.  One  would  like  to  know  more  about 
how  information  that  is  meaningful  to  an  individual  is 
stored  in,  and  retrieved  from,  long-term  memory  for  use  in 
tasks  that  are  really  significant  for  the  individual. 

21.  The  effect  of  context  on  memory  is  obviously  important 
but  not  well-understood.  One  may,  for  example,  be  able  to 
remember  the  combination  of  a  lock  with  the  lock  in  hand, 

but  not  otherwise.  Other  examples  of  the  importance  of 
context:  remembering  a  musical  score,  how  to  get  from  one 

place  to  another,  how  to  accomplish  an  athletic  maneuver. 

22.  It  was  reported  that  people  who  have  to  insert 
data  into  military  computer  systems  often  have  difficulty 

in  remembering  input  codes.  Error  rates  as  high  as  30%  have 
been  obtained  in  some  data  input  studies. 
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23.  It  was  reported  that  doctors  typically  select  all 

of  their  prescription  drugs  from  a  very  small  set  of  alternatives. 
(The  point  is  not  that  all  doctors  use  the  same  small  set  but 
that  each  doctor  has  his  own  favorite  drugs.)  The  question  is 
whether  this  is  a  result  of  memory  limitations  (i.e.,  the  doctor 
only  remembers  a  few  drugs)  or  the  result  of  a  rational  memory 
process  in  which  he  has  examined  a  larger  set  and  has  decided 
that  the  smaller  set  is  adequate. 

24.  Another  general  problem  that  was  noted  is  that  of 
combining  effectively  information  that  is  stored  in  human  memory 
with  information  that  is  stored  in  a  computer  data  base. 

25.  With  respect  to  large  data  bases  that  are  intended 

to  provide  up-to-date  information  for  a  user  on  some  particular 
topic,  several  problems  were  identified.  Among  them  are  the 
following: 

a.  The  problem  of  data  capture.  Computer-based  informa¬ 
tion  systems  that  work  well  (e.g.,  airline  ticketing  systems, 
rental  car  systems,  hotel  reservation  systems,  etc.)  have 
the  property  that  all  transactions  take  place  via  the 
computer  system.  That  is  to  say,  one  cannot  get  a  seat  on 
a  plane,  or  reserve  a  rental  car  or  a  hotel  room  without 
going  through  the  system.  Consequently,  barring  malfunctions, 
one  can  always  assume  that  the  computer  system  has  the 
latest  information  vis-a-vis  commitments  and  available 
resources.  Any  system  that  has  to  rely  on  voluntary  inputs 
for  data  update  is  probably  doomed  to  failure. 
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b.  The  problem  of  searching  and  retrieving  information 
from  a  large  data  base. 

c.  The  problem  of  discovering  from  the  data  base  the 
interesting  relationships  that  are  represented  only 
implicitly . 

26.  Another  line  of  research  that  could  yield  useful 
results  is  the  study  of  retrieval  cues.  How  can  memory  best 
be  probed  for  information  that  is  assumed  to  be  there  but 
proves  to  be  difficult  to  retrieve? 

27.  In  a  discussion  of  verbal  memory,  it  was  pointed  out 
that  people  seem  to  be  able  to  remember  nouns  better  than  verbs. 
The  errors  that  are  made  in  verb  recall  are  often  synonyms 

for  the  verbs  that  are  not  recalled  correctly.  One  theory 
explains  this  by  postulating  that  the  meaning  of  a  verb  is 
represented  largely  by  its  effect  on  a  noun. 

28.  It  was  pointed  out  that  certain  types  of  intelligence 
analysts  seem  to  file  information  under  the  category  "country," 
and  that  they  use  about  100  subcategories  per  country.  There 
are  specialists  for  countries  and  for  topical  areas  such  as 
political,  economic,  etc.  In  attempting  to  keep  their  own 
data  bases  current,  they  search  documents  for  specific  indicators. 
In  particular,  they  look  for  low-probability  events  that  would 
tend  to  indicate  significant  developments.  Special  attention 
is  given  to  descriptors  that  would  be  good  retrieval  cues: 
dates,  names,  etc.  Attention  is  also  given  to  the  source  of 
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any  item  of  information,  and  the  information  is  classified 
in  terms  of  the  level  of  its  source.  Judgments  must  be  made 
concerning  reliability,  concerning  what  information  to  retain, 
and  what  to  discard.  Analysts  seem  to  have  pretty  good  ideas 

about  what  they  might  want  later  and  why. 

A  common  memory  failure  seems  to  be  not  adding  up  all  the 

indicators  that  one  has,  even  when  they  are  all  known  by  a 
single  analyst.  A  question  that  arises  is  whether  computers 
might  help  on  this  problem.  They  clearly  can  help  on  the  problem 
of  cross-referencing  documents.  Might  they  also  help  on 
deciding  what  data  are  significant  to  any  particular  issue 

or  decision? 

29 .  It  was  noted  that  intelligence  agents  sometimes  have 
a  problem  of  forgetting  the  problem  that  they  are  working  on. 

30.  The  sorting  of  information  into  nondisjunctive 
categories  (Russian  reconnaissance,  evacuation  plans,  etc.) 
was  also  noted. 

31.  A  distinction  was  made  between  operations  that  can 
be  performed  on  knowledge  and  operations  can  be  performed  on 


data. 

32.  A  general  problem  of  any  system  or  operation  that 
must  make  use  of  data  from  more  than  one  data  base  is  that 
of  integrating  the  information  that  comes  from  the  various  data 
bases.  The  problem  is  likely  to  be  particularly  severe  if  the 
different  data  bases  were  organized  for  different  purposes. 
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33.  The  notion  of  an  "expert"  software  module  was 
mentioned  in  connection  with  information  retrieval.  An  expert 
module  is  a  software  procedure  that  knows  how  to  answer  a 
specific  type  of  question  from  a  large  data  base. 

34.  A  human  factors  problem  relating  to  human  interaction 
with  large  data  bases  and  question-answering  systems  is  that 

of  defining  acceptable  interaction  dynamics.  How  often,  for 
example,  will  users  tolerate  the  response  "Don't  know,"  or 
"Please  rephrase  your  question?" 

35.  There  is  a  need  for  a  better  understanding  of  memory 
for  structured  descriptions.  It  was  pointed  out  that  chess 
masters  have  a  better  ability  than  others  to  remember  the 
arrangement  of  chess  pieces  on  a  board,  but  this  is  true  only 
if  the  arrangement  is  a  legitimate  one  and  not  otherwise. 

36.  The  point  was  made  that  our  understanding  of  memory 
is  likely  to  be  limited  until  we  have  a  better  understanding 
of  what  determines  conceptual  complexity. 

37 .  The  assertion  was  made  that  a  user  of  a  data  base 
should  not  have  to  understand  the  way  in  which  the  data  are 
structured  in  order  to  use  the  data  base  effectively.  In 
keeping  with  this  notion,  a  system  was  conceived  in  which  there 

are  three  major  components: 
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a.  A  system  interface  which  represents  the  point 
of  contact  between  system  and  user 

b.  A  data  module 

c.  An  intelligent  module  that  contains  knowledge 
about  both  the  system  interface  and  the  data,  and 
provides  a  link  between  them. 
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WORKSHOP  OM  STRESS  EFFECTS  OR  o 


SYSTEM  OPERATION 


The  following  are  representative  of  the  types  of  issue, 
that  we  hope  to  discuss  at  the  vrorkshop  in  Los  Angeles 
30-31  August.  You  are  invited  to  add  to  the  list. 


1) 

2) 

3) 

>0 

!3) 

6) 

7) 

8) 

9) 

10) 

11) 

12) 

1?) 


rnn  v.hat  ciepree  to  experiential,  personality,  and  physio- 
Sslcai  factors ■ determine  susceptibility  or  resistance 
to  stress  effects'? 

Can  stable,  long  term  personaiity  traits  be  identified 

which  mitigate  effects  of  stress  on  individuals. 

Can  similar  physiological  traits  be  identifieo?  . 

Can  such  personality  or  physiological _ traits  (if  they 
exist)  be  acquired  through  formal  training. 

Can  individuals  possessing  such  traits  be  identified? 

-it  non  i  Vin  behavior  of  particular  individuals  under 
sSesf L  precUctcd  5,  personality  or  physiological  measures? 

What  tasks  require  what  levels  of  stress  for  optimal  per- 
f ormance? 

Can  a  classification  of  tasks  by  optimal  stress  levels 
be  developed  systematically? 

How  are  specific  tasks  affected  by  extreme  levels  of  stress? 

Which  C3  system  tasks  are  most  ant)  least  susceptible  to 
degradation  by  stress.'’ 

,  .  .onopf<5  0'  C3  tasks  be  designed  in  such 

rmanner’ls  tollnimg.1.  the  ’stress  to  which  people  in  C3 
systems  are  subjected i 

Can  psychotropic  (mood  modifying)  drugs  reduce  stress  in 
such  settings? 

Can  the  social  setting  of  task  performance  be  arranged  to 
La  .  ;  tre-s  effects,  as,  for  example,  by  sharing 

responsibility ,  well  timed  relief  schedules,  organrsa- 
tional  structures,  etc.? 


I'D 

15) 


Are  special  stress-relieving 
crisis  management? 


techniques 


Hov;  can  commanders  and  other  users  of  C 
nize  situations  in  which  stress  degrade 
Are  such  situations  predictable? 


useful  for 


J  systems  recog- 
s  effectiveness? 


16)  What  activities  relieve  stress  while  off-duty? 

37)  Can  motivational  factors  be  employed  to  increase  indi¬ 
vidual  or  group  resistance  to  stress? 

18)  What  are  the  correlations  among  the  various  measures 
of  stress  in  common  use? 


19) 


Can  a  single 
on  the  basis 


>  superior  measure  be  devised  or  selected 
of  reliability  and  universality? 


20)  Can  any  measure  oi’  stress  support  quantitative  predic¬ 
tions  of  the  magnitude  of  stress  effects? 


21)  Are  communications  upward  and  downward  in  the  chain  of 
command  similarly  affected  by  stresses? 


SYNOPSIS  0?  C3 »-  STRESS  CONFERENCE 


Twenty  one  persons  attended  the  Los  Angeles  Conference 
on  August  30  and/or  31  in  one  capacity  or  another.  Names 
and  addresses  of  those  not  associated  with  BBN  and  ARPA  are 
attached.  Participants  made  informal  presentations  during 
which  they  discussed  the  relevance  of  research  with  which 
they  were  familiar  to  stress  problems  in  C3  systems,  and 
raised  issues  which  appeared  to  be  of  importance  for  stress 
research  in  general.  No  attempt  was  made  to  define  the  term 
"stress’1 ;  indeed,  over  a  score  of  specific  stressors  were 
discussed  in  contexts  ranging  from  industrial  safety  and  pro¬ 
duction,  through  long  term  occupational  exposure,  to  societal 
organization.  Some  of  the  viewpoints  expressed  on  the  more 
salient  issues  are  summarized  in  the  following  sections. 


1.  Relative  Importance  of  Chronic  vs.  Acute  Stress 
in  Systems 


The  view  was  expressed  that  research  on 

3 

acute  stresses  was  most  germane  to  human  performance  in  C 
systems.  Subsequent  discussion,  although  generally  in  agree¬ 
ment  with  this  view,  also  made  it  clear  that  an  individual  s 
ability  to  cope  with  acute  stress  varies  with  levels  of  chronic 
stress.  Thus,  for  example,  a  commander’s  ability  to  deal 
with  an  emergency  situation  depends  not  only  upon  the  imme¬ 
diate  stresses  to  which  the  commander  is  subjected,  but  also  to 
the  commander’s  experience  in  handling  prior  stressful  situations 
and  current  levels  of  exposure  to  long  term  stresses  unrelated 
to  the  immediate  emergency.  It  was  generally  agreed  that  iso¬ 
lated  study  of  acute  stress  effects  would  be  of  limited  benefit. 


2.  "Realism"  in  Stress  Research 


The  topic  of  "realism"  in  stress 
several  contexts.  In  the  context  oi 
experiments  to  crisis  situations  in  C 


research  was  raised  in 
extrapolation  of  results  of 
3  systems ,  many  participants 


J 


were  skeptical.  Their  skepticism  seemed  based  primarily  upon 
the  disparity  of  levels  of  stress  that  could  be  produced  in 
the  laboratory  and  those  to  which  operational  personnel  might 
be  exposed. 

In  the  context  of  the  ethics  of  stress  research,  the  issue 
of  "realism"  was  seen  as  a  substantial  problem  to  the  extent 
that  it  required  deception  of  test  subjects.  Potential  means 
of  circumventing  rules  for  protection  of  human  subjects  were 
denounced  by  some  on  ethical  grounds ;  others  pointed  out  that 
even  the  more  extreme  forms  of  experimental  stress  popular 
years  ago  produced  little  in  the  way  of  substantive  findings. 

It  was  generally  agreed  that  the  hope  for  future  research 
in  this  area  lay  in  carefully  designed  field  experiments.  The 
favored  paradigm  seemed  to  be  long  term  prospective  study  of  a 
population  exposed  (for  extra-scientific  reasons)  to  varying 
levels  of  actual  stresses.  Difficulty  of  access  to  such  popu¬ 
lations  was  discussed  at  length.  A  number  of  military  situa¬ 
tions  which  offered  such  opportunities  were  suggested. 

3.  Means  of  Alleviating  Stress 

A  number  of  means  of  minimizing  the  effects  of  stress  on 
performance  in  C3  systems  were  discussed.  Considerable  attention 
was  given  to  the  social  context  in  which  people  are  exposed  to 
stress.  Social  supports  for  coping  behavior  were  contrasted 
among  various  organizational  settings ;  knowledge  of  coworkers 
performance  and  workload  were  discussed  from  various  perspectives 
and  scheduling  of  relief  personnel  and  shift  length  optimi¬ 
zation  were  proposed  as  potential  stress  relief  techniques. 

Other  schemes,  ranging  from  intentional  witholding  or  delaying 
of  information  to  engaging  in  activities  that  dissipate  effects 
of  stress  were  also  mentioned  on  several  occasions. 

4.  Actual  vs.  Perceived  Stress  Levels 

An  important  distinction  was  made  between  objective  and 
subjective  measures  of  stress.  In  most  cases,  it  is  the  tit 


of  the  workload  to  an  Individual's  expectations  of  performance 
abilities  which  determine  the  amount  of  stress  experienced 
Thus,  it  may  he  difficult  to  infer  dosage-response  relation 
ships  from  research  that  applies  the  same  objective 
stress  to  different  individuals. 

5.  Reserve  Capacity  Model 

A  common  finding  in  stress  research  is  that  test  subjects, 
if  sufficiently  motivated,  are  capable  of  maintaining  per¬ 
formance  in  the  face  of  severe  stress  for  some  time.  Both 
anecdotal  evidence  and  experimental  results  of  various  sorts 
Quoted  at  the  conference  in  support  of  this  find  ng. 

There  was  also  some  discussion  of  recent  findings  that  P >  “ 
formance  decrements  may  occur  after  the  application  of  stresso 
has  ceased.  The  inference  was  drawn  that  coping  has  a  cost 
which  may  be  deferred,  but  which  eventually  must  be  pa  • 

A  "reserve  capacity" ' model  was  discussed  to  account 
these  findings.  Although  proposed  in  various  ™ 

stock  of  psychic  energy",  “finite  energy  r“s  ,  etc.) 
may  be  simply  conceptualized  in  an  hydraulic  analogy.  A 
of  fluid  (representing  a  supply  of  whatever  resources  are 

1  1s  drained  at  a  certain  rate  by  coping  with 

needed  for  coping)  is  arameu  aw  _ 

stressful  situations.  It  is  refilled  at  same  rate  (and  at  so 
oost)  at  less  stressful  times.  Experimentation  directed  at 
t  • fi  cation  of  the  dynamics  of  such  a  model  would  seem  to 
highly  useful.  First,  factors  that  affect  the  capacity,  draining, 

and  recharging  of  the  tank  must  be  identified.  Hextmeof 
quantifying  such  factors  in  common  terms  must  ~ 
rates  of  expenditure  and  replenishment  must  be  established. 
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MEMORANDUM 


To: 

From: 

Subject: 


Invitees  to  Workshop  it 5:  Decision  Making  and 
Raymond  S.  Nickerson,  Bolt  Beranek  and  Newman 
Workshop 


Problem  Solving 
Inc . 


Date:  25  August  1976 


We  are  engaged  in  a  project  for  the  Human  Resources  Office  of  the  Defense 
Advanced  Research  Projects  Agency  on  Human  Factors  in  Command,  Control 
and  Communication  (C3)  systems.  A  major  objective  of  the  project  is  the 
development  of  a  plan  for  a  program  of  research  on  this  topic.  It  is  not 
our  purpose  to  design  C3  systems,  but,  rather,  to  identify  researchable 
human  factors  problems  relating  to  the  performance  of  such  systems. 

As  one  means  to  this  end,  we  are  convening  a  series  of  workshops  to  bring 
together  small  groups  of  appropriate  individuals  to  discuss  various  aspects 
of  the  problem.  One  such  meeting  is  scheduled  to  be  held  in  Cambridge, 

Mass.  (BBN  office  at  10  Moulton  Street)  on  September  21,  22,  1976.  The 
purpose  of  this  meeting  will  be  to  discuss  decision  making  and  problem 
solving  in  relationship  to  C3  systems.  In  particular,  what  we  hope  to 
accomplish  is  the  articulation  of  some  significant  researchable  problems 
pertaining  to  this  topic  in  C3  contexts. 

This  memo  is  a  follow-up  to  the  invitation  you  recently  received  by  phone 
to  participate  in  this  meeting.  We  are  very  pleased  that  you  have  agreed 
to  meet  with  us  and  look  forward  to  seeing  you  on  the  21st.  We  have 
limited  the  number  of  participants  to  not  more  than  20  so  we  can  maintain 
a  lively  discussion  in  an  informal  atmosphere. 

What  we  want  at  the  meeting  is  a  far-ranging  discussion  of  issues  that 
participants  consider  to  be  germane,  to  the  general  topic.  To  provide  a 
context  and  some  points  of  departure,  we  have  drafted  a  brief  background' 
paper  on  C3  systems,  and  prepared  a  list  of  questions  that  are  repre¬ 
sentative  of  those  that  we  might  discuss  at  the  meeting.  Both  of  these 
documents  are  enclosed. 

We  plan  to  devote  the  first  morning,  or  whatever  portion  of  it  is  required, 
to  brief  (10-15  minute)  presentations  by  participants.  While  the  invitation 
is  not  contingent  on  your  agreeing  to  make  such  a  presentation,  we  would 
appreciate  it  very  much  if  you  would  be  willing  to  take  this  time  to  give 
your  perspective  on  the  general  problem.  If  you  wish,  you  might  use  the 
time  to  introduce  possible  topics  that  you  feel  are  especially  important 
to  explore  during  the  meeting. 

The  meeting  will  start  at  9:30  a.m.,  Tuesday,  September  21,  and  end  early 
in  the  afternoon  of  September  22.  If  you  would  like  us  to  make  hotel 
reservations  for  you,  please  indicate  that  on  the  enclosed  self-addressed 
postcard  (specify  the  night  or  nights) ,  and  return  it  to  us  as  soon  as 
possible.  Enclosed  is  a  map  of  the  area  to  help  you  find  your  way  to  BBN. 
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From  Raymond  S.  Nickerson 
25  August  1976 
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Probably  the  easiest  way  to  get  here  from  the  airport  is  to  take  a  cab. 

If  the  driver  does  not  know  where  Moulton  Street  is,  tell  him  it  is  in 
the  Fresh  Pond  area  of  Cambridge.  We  will,  of  course,  reimburse  you  for 
reasonable  expenses  incurred  as  a  result  of  your  participation  (travel, 
meals,  lodging)  and  can  offer  you  an  honorarium  of  $150  per  day.  Perhaps 
the  more  significant  motivation  for  participation  is  the  chance  to  help 
shape  a  new  DARPA  program  that  is  likely  to  provide  research  opportunities 
for  human  factors  specialists  in  the  future. 

See  you  on  the  21st. 


Sincerely, 


Raymond  S.  Nickerson 
f  tm 


Enclosures: 

Background  paper 
List  of  questions 
Self-addressed  stamped  postcard 
Map 
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Workshop  if  5:  Decision  Making  and  Problem  Solving 

The  following  are  representative  of  the  types  of  questions  that  we 

would  hope  to  discuss  at  the  workshop.  You  are  invited  to  add  to  the  list. 

1.  How  do  the  decisions  made  by  groups  differ  from  those  made  by  individuals? 
What  prescriptive  frameworks  exist  for  judging  group  decision  making? 

2.  What  heuristics  are  used  by  problem  solvers  and  decision  makers  to 
infer  states  of  the  world?  What  technique  might  be  employed  to  help  the 
decision  maker  understand  when  these  heuristics  are  appropriate  and  when  they 
are  inappropriate? 

3.  A  characteristic  of  most  information  systems  is  that  occasionally 

they  present  the  problem  solver /decision  maker  with  data  that  are  in  error. 
What  methods  are  available  or  might  be  developed  to  aid  the.  user  of  such  data? 

4.  Many  taxonomies  and  discussions  of  problem  solving  and  decision  making 
are  episodic  in  nature— that  is  to  say,  they  view  the  activities  as  being 
comprised  of  sequences  of  subactivities  (information  seeking,  data  aggre¬ 
gation,  hypothesis  testing,  etc.).  Does  this  approach  present  an  adequate 
model  of  the  command  and  control  function?  What  alternative  models  might 
be  formulated  to  highlight  the  inevitable  interactions  among  these  sub¬ 
activities? 

5.  Most  practical  problems  require  that  a  set  of  actions,  rather  than  a 
single  action,  be  taken  for  a  complete  solution.  Frequently,  however,  one 
or  more  members  of  the  set  cannot  be  defined  until  a  prior  member  has  been 
identified.  How  might  the  decision  maker  be  aided  in  selecting  a  consistent 
set  of  actions  and  in  identifying  critical  options  as  he  proceeds? 

6.  How  might  an  interactive  system  be  configured  to  help  the  commander 
recognize  similarities  and  dissimilarities  between  a  current  problem  and 
one(s)  he  has  faced  in  the  past?  How  might  that  system  aid  him  to  select 
an  appropriate  response? 
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7.  What  models  of  human  decision  making  and/or  problem  solving  currently 
exist  in  a  form  suitable  for  inclusion  in  a  simulation  of  command,  control 
and  communication  processes? 

8.  What  decision  aids  and/or  problem  solving  aids  exist  that  could  be 
used  to  assist  the  commander  or  lower  ranking  personnel  engaged  in 
activities?  What  would  need  to  be  done  to  these  aids  to  make  them  opera¬ 
tionally  suitable  for  use? 

9.  What  practical  techniques  exist  for  planning  and  for  prediction  of 
outcomes?  How  might  these  techniques  be  effectively  implemented  in  an 
environment  employing  interactive  displays? 

10.  What  techniques  exist  for  capture  and  modeling  of  the  inferential 
and  decision  processes  of  problem  solvers?  Could  these  techniques  be 
utilized  in  an  interactive  system  to  aid  and  guide  the  activity  of  the 
commander? 

11.  Much  of  command  and  control  consists  of: 

a)  carrying  out  plans,  and 

b)  elaborating  or  modifying  the  plans  as  the  situation  develops. 

What  is  known  about  human  strengths  and  weaknesses  in  those  two  activities? 

12.  How  might  a  computer  assist  a  human  operator  in  performing  those 
activities?  How  might  a  human  operator  assist  a  computer? 

13.  How  does  the  individual's  conception  of  the  problem  bias  his  search 
for  relevant  data? 

14.  How  does  the  organization  of  data  affect  the  individual's  process 
of  hypothesis  generation. 

15.  What  kinds  of  systematic  individual  differences  are  found  in  problem 
solving/decision  making  strategies? 
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16.  To  what  extent  can  decision  making  skills  be  taught  independently 
of  specific  decision  problems?  That  is  to  say,  to  what  extent  can  such 
skills  be  taught  in  such  a  way  that  they  generalize  across  a  variety  of 
decision  situations? 

17.  Investigators  have  discovered  a  number  of  ways  in  which'decision  makers 
tend  to  be  biased  in  their  decisions.  How  can  these  biases  be  used  to 
advantage  in  decision  making  systems? 

18.  A  distinction  can  be  made  between  a  fundamental  human  limitation  and 

a  tendency  on  the  part  of  an  individual  to  perform  in  a  suboptimal  fashion, 
even  when  he  has  the  capability  of  performing  more  nearly  optimally.  How 
useful  might  this  distinction  be  in  classifying  decision  making  performance 
and  .in  developing  methods  for  compensating  both  for  human  limitations  and 
for  stereotyped  ways  of  performing  suboptimally? 

19.  How  does  one  distinguish  between  reasons  for  a  decision  and  after-the- 
fact  rationalizations? 

20.  How  can  one  maximize  the  probability  that  incoming  information  will 
be  assessed  objectively  and  without  bias  by  the  decision  maker  even  after 
he  has  made  a  commitment  in  favor  of  some  hypothesis  about  the  state  of  the 
world? 


MEMORANDUM 


Subject: 


Participants  in  Workshop  on  . 

Decision  Making  and  Problem  Solving 

r.  s.  Nickerson 

Notes  from  meeting 


3  November  1976 


Enclosed  is  a  set  of  notes  from  the  workshop  on 
Decision  Making  and  Problem  Solving  in  C3  Systems  that 
was  held  at  BBN  on  September  21,  22,  1976.  We  have  made 
no  effort  to  produce  "minutes"  of  the  meeting,  or  even  to 
organize  the  notes  in  any  particular  way.  We  simply 
jotted  down  the  points  that  were  made,  more  or  less  in 
the  order  in  which  they  were  made  at  the  meeting.  Some  of 
them  will  undoubtedly  prove  to  be  a  bit  cryptic;  however, 
they  may  serve  to  complement  or  supplement  your  own  notes. 

I  want  to  thank  you  for  attending  the  workshop  and 
for  your  contribution  to  the  discussions.  I  hope  you  found 


it  of  soma  value. 


Bolt  Beranfck  find  Nowrnan  Iric. 


NOTES  FROM  SEPT. 


21,22  MEETING  RE  DECISION  MAKING  AND 


PROBLEM  SOLVIt 


1.  Many  decision-aiding  techniques  require  that  users  be 
able  to  think  in  probabilistic  terms.  They  must,  for  example, 
be  able  to  evaluate  possible  states  of  the  world  in  terms  of 
their  relative  likelihoods  of  actualization. 

The  point  was  made  that  people  may  differ  considerably  in 
their  ability  to  understand  or  use  probabilistic  notions.  In 
particular,  it  was  suggested  that  there  may  be  significant  cross- 
cultural  differences  in  this  regard,  and  some  data  were  described 
to  substantiate  this  view. 

Systematic  cross-cultural  differences  in  the  degree  to 
which  people  can  think  in  probabilistic  terms  could  turn  out  to 
be  an  increasingly  significant  issue  if  command  and  control 
operations  of  the  future  tend  more  to  involve  the  interaction 
of  people  of  various  nationalities  and  cultures  with  a  common 
system.  The  assumption  that  probabilistic  terms  mean  the  same 
things  to  people  of  various  culture,  if  in  fact  they  do  not, 
could  lead  to  unfortunate  results. 


2.  Some  data  were  reported  on  the  question  of  how  accurately 
people  can  judge  their  own  knowledge.  In  this  regard,  the  notion 
of  an  uncertainty  calibration  curve  was  discussed.  An  uncertainty 
calibration  curve  shows  how  the  degree  of  confidence  an  individual 
expresses  in  an  assertion  relates  to  the  probability  that  the 
assertion  is  correct.  For  an  optimally  calibrated  person,  this 
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curve  would  have  a  slope  of  1,  inasmuch  as  degree  of  confidence 
would  predict  accuracy  perfectly.  For  example,  considering  all 
judgments  for  which  an  individual  was  willinci  to  say  that  the 
probability  of  his  being  correct  was  .8,  805  of  them  should  be 
correct. 

Uncertainty  calibration  curves  almost  invariably  have  a 
slope  of  less  than  1.  There  was  some  discussion  of  the  factors 
that  appear  to  affect  the  slope  and  intercept  of  this  curve 
and,  in  particular  of  the  fact  that  efforts  to  train  people  to 
calibrate  more  accurately  have  been  generally  unsuccessful. 

A  question  of  some  interest  that  seems  not  to  have  been 
resolved  yet  is  whether  people  who  know  more  (have  more  facts  at 
their  disposal)  are  also  better  able  to  assess  their  knowledge 
(have  more  nearly  optimal  uncertainty  calibration  curves) .  It 
was  agreed  that  this  area  is  one  that  could  profit  from  more 
research. 

3.  This  point  relates  to  both  of  the  preceding  ones. 

Some  data  were  reported  that  suggest  that  Chinese  people  tend 
to  have  a  discontinuity  in  their  uncertainty  calibration  curves. 
More  specifically,  the  curve  is  relatively  flat,  except  at  the 
upper  extreme  (i.e.,  where  confidence  approaches  certainty)  at 
which  point  the  curve  turns  upward.  This  finding  supports  the 
notion  that  Chinese  people  tend  not  to  see  the  world  in  terms 
of  probabilities  but  in  a  more  dichotomous  fashion. 
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4.  A  distinction  was  made  between  the  technology  available 
for  management-decision  problems  and  that  for  real-time  decision 
aids.  It  was  suggested  that  the  former  technology  is  years 
ahead  of  the  latter . 

5.  A  decision  aid  involving  the  decomposition  of  a  problem 
into  subproblems  that  could  be  solved  independently  with  the  use 
of  regret  matrices  was  described.  The  need  for  constraints  and 
tree— pruning  procedures  to  contain  complex  decision  problems 
within  manageable  limits  was  noted. 

6.  An  important  capability  for  any  general-purpose 
(jQcislon  aid  is  that  of  performing  sensitivity  analyses.  If 
one  cannot  determine,  with  a  fair  degree  of  accuracy,  the 
relative  seriousness  of  various  types  of  errors  that  are 
possible,  one  does  not  have  a  firm  basis  for  deciding  how  much 
it  is  worth  to  attempt  an  optimal  solution  to  a  decision  problem 
as  opposed  to  a  subopt.imal  but  readily  available  one. 

7.  The  following  question  was  discussed  at  some  length: 

When  is  it  best  to  use  experts'  judgments  rather  than  a  decision¬ 
making  algorithm?  It  was  pointed  out  that  in  order  to  apply 

a  decision-making  algorithm,  one  must:  (a)  choose  dimensions 
in  terms  of  which  the  decision  is  to  be  made,  (b)  code  these 
dimensions,  and  (c)  integrate  the  dimensional  information  that 
is  to  be  used.  Studies  comparing  the  effectiveness  of  statistical 
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versus  clinical  predictions  have  tended  to  show  statistical 
comparisons  to  advantage.  It  was  suggested  that  experts  should 
be  used  to  choose  and  code  dimensions,  and  that  algorithms 
should  be  used  to  integrate  the  dimensional  information. 

8.  The  role  of  pattern  recognition  at  various  levels  of 
decision  making  was  stressed.  This  includes  the  recognition 
of  patterns  that  can  be  formed  by  applying  allowed  operations 
to  other  patterns.  The  importance  of  pattern  perception  in 
the  performance  of  chess  experts  was  noted  as  an  interesting 
analogy.  It  was  pointed  out  that  the  patterns  in  this  case 

are  not  only  existing  patterns  but  patterns  that  can  be  obtained 
as  a  result  of  admissible  operations  (moves)  on  existing  patterns. 
The  patterns  (to  be  realized)  represent  subgoals  in  the  problem 
solution . 

9.  It  was  suggested  that  there  was  something  of  value  to 

be  learned  from  the  study  of  the  rules  of  evidence  and  juris- 

3 

prudence  that  could  be  applied  to  C  decision  problems.  One 

3 

significant  way  in  which  jurisprudence  and  C  decisions  are 
similar  is  in  the  fact  that  information  is  provided  in  both 
cases  for  a  variety  of  reasons,  including  deception. 

10.  It  was  also  noted  that  the  study  of  jury  decisions 

was  one  approach  to  the  study  of  group  decision  maing  in  general. 
Some  data  were  referred  to  that  suggest  that  the  deliberation 
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process  by  which  the  jury  arrives  at  a  decision  may  differ 
depending  on  the  decision  ruie  (i.e.,  whether  the  decision  is 
to  be  based  on  majority  opinion,  two-thirds'  opinion,  etc.,. 

U.  in  connection  with  jury  decisions,  evidence  was  cited 

f,~,v  "facts"  that  tend  to  confirm  a  decision 
of  selective  memory  for  facts 

that  has  been  made.  There  seems  to  be  a  question  regarding 
such  findings  as  to  whether  the  effect  is  a  true  memory  effect 
or  whether  it  results  from  a  better  initial  assimiliation  of 

the  confirming  facts  which  could  have  been  instrumental  in 

4  Vam  fircf  n  1  a  c  e  (cart-horse  problem), 
producing  the  decision  in  the  first  place 

12.  It  was  noted  that  a  distinction  should  be  made  between 
habit-following  and  decision-mating,  and  that  many  of  the  "acts 
o£  choice"  that  pass  for  decision-mating  really  are  habit-driven. 
An  example  is  the  purchasing  of  goods,  as  for  example,  brand-name 

canned  goods  in  a  grocery  store.  Selection  of  Brand  X  canned 

V,  *  rart  vriD  do=s  not  necessarily  constitute 
peas  on  any  given  shopping  tr  I 

a  decision  to  buy  Brand  X  over  alternative  brands.  IL  may  mv 
simply  carrying  oat  habitual  behavior.  This  may  have  both 
desirable  and  undesirable  implications.  On  the  plus  side  is  the 
efficiency  of  habitual  behavior-,  it  obviates  spending  the  trme 
and  energy  that  would  be  required  to  make  an  independent  decision 
for  every  action.  The  negative  implication  is  that  it  sometimes 
leads  to  stereotyped  and  maladaptive  behavior.  The  point  is 
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well  taken  that  this  may  have  both  desirable  and  undesirable 
implications . 

good:  efficient — too  much  time  and  work  to  make  real 

decisions  about  everything 

bad:  stereotyped,  unadaptable  decision-making 

13.  A  common  theme  in  much  decision-making  research  is 
man's  limitations  as  a  decision  maker.  The  multidimensional 
judging  of  livestock  by  experts  in  an  agricultural  college  was 
used  as  an  example  of  how  good  people  can  be  at  decision  making. 
Some  evidence  was  cited  that  judges  used  as  many  as  8  to  11 
uncorrelated  dimensions  in  judging  the  quality  of  livestock. 

Why  they  seem  to  do  so  badly  with  correlated  dimensions  is 
unclear . 


14.  A  general  challenge  to  people  who  would  develop 
problem-solving  and  decision-making  aids  is  to  find  ways  of 
getting  out  of  people's  heads  what  they  know  (but  may  not  know 
they  know)  that  is  relevant  (but  may  not  be  spontaneously 
perceived  as  relevant)  to  the  problem. 

15.  Some  observations  were  made  about  the  possible  role 
of  cognitive  style  in  decision-making  performance.  The  notion 
was  that  there  may  be  systematic  differences  between  people 
that  can  be  characterized  in  terms  of  a  few  dimensions,  e.g., 
abstract-concrete ,  passive-active,  logical-intuitive. 
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16.  It  was  suggest ed  that  automated  techniques  seldom  can 
be  introduced  smoothly  into  existing  organizational  structures. 

Their  introduction  almost  invariably  necessitates  changing  the 
structures. 

17.  There  was  some  discussion  on  the  question  of  whether 
decision  aids  have  to  be  designed  explicitly  for  specific  situations 
in  order  to  be  useful.  Aids  that  could  be  applied  effectively 
across  a  variety  of  situations  obviously  would  be  of  greater 
interest  than  those  that  can  ce  applied  only  to  a  specific  problem. 
There  did  not  seem  to  be  a  consensus  that  general-purpose  aids 

were  possible,  given  the  current  state  of  the  art  and  the  populace. 

18.  it  was  pointed  out  that  one  of  the  major  benefits  of 
attempting  to  analyze  a  decision  situation  to  the  point  that  a 
decision  model  could  be  applied  to  it  is  the  better  understanding 
of  the  problem  that  the  decision  maker  may  get  as  a  result  of 
attempting  to  do  the  analysis.  This  is  independent  of  whether 
the  model  or  aid  proves  to  be  usable  or  not. 

19.  The  importance  of  recognizing  the  hierarchical  goal 
structure  of  any  complex  organization  was  emphasized.  It  is 
particularly  important  to  recognize  that  the  goals  at  one  echelon 
may  be  different  from  those  operating  at  another  echelon.  It  is 
imperative  to  the  effective  functioning  of  any  complex  system 
that  the  goals  at  different  echelons  at  least  be  consistent  with 
each  other,  and  that  those  at  the  lower  echelons  bo  supportive  of 
those  at  the  higher  ones. 
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20.  The  point  was  made  that  increasing  amounts  of  data  are 
being  accrued  on  the  way  people  behave  in  decision-making  sit  na¬ 
tions,  but  that  this  research  should  be  extended  in  two  directions: 
first,  there  is  a  need  for  a  mechanism  through  which  research 
findings  can  be  applied  to  real  decision-making  problems;  second, 
having  determined  the  situations  in  which  biases,  errors,  irration¬ 
alities  and  general  foolishness  tend  to  occur,  there  is  a  need 
for  psychologists  in  the  fields  of  memory  and  cognition  to  try 
to  discover  the  psychological  processes  underlying  these  tendencies. 

21.  The  problem  of  cost-benefit  incompatibilities  was  dis¬ 
cussed.  The  National  Health  Insurance  was  used  to  illustrate 
this  point:  People  are  willing  to  support  NHI  only  if  offered 
at  reasonable  cost;  however,  people  see  the  primary  (perhaps  only) 
worthwhile  benefit  of  NIII  to  be  in  the  coverage  of  small  medical 
fees,  such  as  visits  to  doctors,  dental  fillings,  etc.  These  two 
stipulations  are  diametrically  incompatible;  despite  this,  the 
resolution  will  probably  be  a  moderately  priced  NHI  program 
which  covers  only  moderately  expensive  costs— a  compromise  which 

will  not  please  anyone  very  much. 

22.  What  stems  to  be  an  inability  of  people  to  deal  ade¬ 
quately  with  risk  assessment  was  discussed.  Despite  governmant- 
snbs.idit.ed  insurance  premiums,  for  example,  it  is  hard  to  convince 
high-risk  flood  populations  that  such  insurance  is  a  bargain. 
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23.  There  was  some  discussion  of  the  construction  of  fault 
trees — contingency  trees  for  which  the  end  branches  correspond 
to  the  aversive  consequences  of  faults,  e.g.,  failure  in  nuclear 
power  plant.  In  constructing  such  trees  there  is  the  problem  of 
how  to  be  sure  that  every  possible  end-state  (consequence)  has 
been  included  and  of  identifying  what  the  consequences  of  over¬ 
sight  might  be. 

24.  It  was  noted  that  there  appears  to  be  a  set  oi  primitive 
relevance  judgments  that  people  tend  to  make  about  the  information 
in  a  problem  statement.  One  tends,  for  example,  to  pay  special 
attention  to  "ifs,"  time  words,  numbers,  set  descriptors,  and 

the  like. 

25.  An  important  aspect  of  problem  solvinq  is  the  ability 
to  classify  problems  as  to  type,  and  to  see  isomorphisms.  When 
people  recognize  a  problem  type,  they  tend  to  apply  a  routine 
problem-solving  frame;  the  difficulty  is  they  are  likely  to  do 
this  whether  or  not  the  application  is  really  appropriate.  On 

the  other  hand,  they  often  fail  to  perceive  an  isomorphism  between 
problems  with  different  surface  representations.  In  these  cases, 
they  systematically  tend  to  solve  what  are  really  the  same  problems 
with  distinctively  different  procedures. 

26.  The  importance  of  the  orner  of  information  presentation 


to  decision  makers  wa 


stressed . 
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27.  Military  intelligence  reports  are  supposed  to  be 
assessed  as  to  their  accuracy  and  source  reliability.  It  was 
claimed  that  analysts  cannot,  or  do  not,  use  both  of  these 
dimensions,  or  at  least  that  to  the  extent  that  they  do,  they 
tend  to  integrate  them. 

28 .  An  analogy  was  drawn  between  scientific  inference 
and  medical  diagnosis;  another  was  drawn  between  juridical  and 
intelligence  proceedings. 

29.  Among  the  things  that  make  problems  difficult  are  the 
lack  of  definition  either  of  goals  or  of  the  problem  space. 

Most  decision  algorithms  require  specificity  in  both  cases  and 
surprisingly  little  attention  has  been  given  to  situations  m 
which  one  or  the  other  is  not  specifiable.  Ill-defined  problems 
are  likely  to  be  especially  prevalent  in  dynamic  situations, 

in  which  the  problem  space  is  constantly  changing,  and  therefore 
are  especially  relevant  to  command  and  control  systems. 

30.  In  studying  or  describing  the  behavior  of  any  man- 
machine  system,  it  is  important  to  distinguish  between  goals  and 
procedures.  More  specifically,  it  is  important  to  distinguish 
between  those  circumstances  in  which  it  is  sufficient  to  provide 
a  component  of  the  system  with  a  goal,  and  leave  it  to  its  own 
devices  to  achieve  that  goal,  and  those  in  which  the  component 
must  be  provided  with  a  procedure. 
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31.  Some  data  were  reported  showing 
.in  which  people  seem  to  misapply  rules  of 


some  systematic  ways 
inductive  inference. 


32.  An  attempt  to  qenerate  a  list  of  known  or  suspected 
limitations  of  human  decision  makers  or  systematic  errors  that 
seem  to  characterize  human  decision  making  produced  the  following 
list.  (Nothing  is  implied  by  the  order  of  items  in  the  list.) 


Affirming  the  consequent.  Applies  to  implicative 
statements.  Tendency  to  assume  that  "if  x  then  Y" 
implies  "if  Y,  then  X." 


-  Failure  to  use  rarity  principle 
i.e.,  given  "if  x,  then  Y,"  the 
an  indicant  of  X  is  Y. 


in  inductive  reasoning, 
rarer  X  is,  the  better 


-  Place-keeping  failures.  This  is  when  your  finger  slips 
out  of  a  book.  4 


-  Suboptimal  use  of  information  re  the  reliability  of  data. 

-  Lack  of  mapping  between  probabilistic  terminology  and 
conversational  terminology. 

-  Biasing  of  frequency  estimates  in  accordance  with 
availability  of  exemplars. 

-  Gambler's  fallacy. 

Compartmentalization  of  data  from  disparate  sources. 

Preference  for  single  orderings- — tendency  to  interpret 
comparative  data  as  transitive. 

-  Overconfidence  in  one's  opinions. 

-  Hindsight  or  retrospective  biases  in  evaluation  and 
analysis . 


Misunderstanding  of  logical  operators.  Boolean  operators,  etc. 

Tendency  to  search  for  data  that  affirm  working  hypothesis 
rather  than  making  critical  test  of  hypothesis.  °  ' 
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-  Misunderstanding  nature  of  expertise. 

-  Effects  of  earlier  commitment  on  subsequent  information 

seekinq  and  interpretation. 

-  Ignoring  base— rate  information. 

-  Illusory  correlations. 

-  Preference  for  redundant  information,  over  independent 

information. 

-  Confusion  of  correlation  with  causation. 

-  Premature  proposal  of  solutions. 

-  Effect  of  winner-loser  status  on  quality  of  decision. 

-  Tendency  towards  inertia  or  status  quo. 

-  Failure  to  perceive  problem  isomorphisms,  analogous 
situations. 

-  Failure  to  perceive  differences  between  situations 
that  are  isomorphic  in  some  respects  but  not  others. 

-  Too  much  credence  in  small-sample  statistics. 

-  Risky-shift  tendency  to  choose  riskier  alternatives 
in  group  setting  than  alone. 

-  Problems  inherent  in  multiperson . filter ,  e.g.,  premature 
filtering,  thresholding,  distortions. 

-  Reluctance  to  be  bearer  of  bad  tidings--especially 
to  boss. 

-  Overcorapensation  for  companions. 

-  Bandwagon  effect. 

-  Binay  fallacy.  Failure  to  recognize  intermediate 

possibilities . 

-  Pluralistic  ignorance.  Failure  to  recognize  ignorance  ^ 
when  widely  shared:  "200  million  people  can't  be  wrong. 

-  Difficulty  of  integrating  or  assimilating  multidimensiont 
information 
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33.  Suggestions  for  future  research  included  the  following: 

a.  Development  of  better  methods  for  doing  sensitivity 
analyses  to  evaluate  the  consequences  of  various  types  of 
decision  errors. 

b.  Experimental  confirmation  of  some  of  the  •  intuitively 
known  facts"  regarding  decision-making  capabilities  and 
limitations  of  human  beings. 

c.  Intensive  studies  of  "decision— making  experts. 

What  is  it  that  characterizes  proficient  decision  makers 
(as  judged  by  their  peers)  from  ineffective  ones?  Storage 
of  massive  amounts  of  knowledge  that  is  relevant  to  the 
decision  problem?  The  application  of  effective  procedures 
for  analyzing  and  solving  decision  problems? 

d.  More  intensive  investigations  of  how  people  structure 
problems  and  decision  situations,  including  how  they 
generate  hypotheses.  The  emphasis  in  the  past  seems  to 
have  been  on  analytic  skills.  There  is  need  to  pay  more 
attention  to  creativity  and  synthesizing  abilities. 

e.  Methods  for  training  of  decision  makers  should  be 
investigated.  There  is  very  little  in  the  literature 
concerning  how  to  train  people  in  decision-making  skills 
in  such  a  way  that  what  they  learn  will  transfer  from  one 
decision-making  context  to  another.  Some  work  has  been 
done  on  debiasing  techniques,  calculated  to  teach  people 
to  compensate  for  some  of  the  systematic  errors  that  have 
been  characteristic  of  decision  makers  in  certain  proba¬ 
bilistic  decision  situations. 

It  is  not.  really  known  at  what  level  of  generality 
decision-making  skills  can  be  taught.  The  question  deserves 
more  attention  from  investigators  than  it  has  received. 


f.  There  is  a  need  for  better  conceptual  frameworks  in 
terms  of  which  to  describe  goals  and  subgoals  of  complex 
systems  and  system  components. 

g.  There  is  a  need  for  more  adequate  theory  to  guide  the 
allocation  of  decision-making  and  problem-solving  functions 
to  men  and  machines  in  man-computer  systems. 

h.  Given  the  continually  decreasing  cost  of  computer 

memory,  it  seems  certain  that  computers  will  be  used  more 

and  more  to  store  massive  amounts  of  data,  to  support 
■  3 

various  C  operations.  It  will,  therefore,  become  increas¬ 
ingly  important  to  find  effective  ways  of  filtering, 
organizing,  searching  and  purging  very  large  data  bases. 

The  storage  of  large  amounts  of  information  will  prove 
to  be  an  encumbrance  and  an  embarrassment  unless  more 
effective  procedures  are  developed  for  utilizing  the 
information  in  such  data  bases. 

i.  Resource  allocation  is  an  example  of  a  generic 

problem  that  all  commanders  face.  Consequently,  a  type 

of  decision  aid  that  could  have  impact  at  many  points 
3 

within  C  systems  would  be  a  resource-allocation  aid. 
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MEMORANDUM 


To: 

From: 

Subject: 


Invitees  to  Workshop  Ko.  6: 
it.  W.  Pew,  Bolt  Beranek  and 
WORKSHOP 


Man-Computer  Dialogue  Development 


Newman  Ine 


•  0^''fes 


Date:  l6  September  1976 


We  are  engaged  in  a  project  for  the  Human  Resources  Office  of  the  Defense 
Advanced  Research  Projects  Agency  on  Human  Factors  in  Command,  Control  and 
Communication  (c3)  systems.  A  major  objective  of  the  project  is  the  development 
of  a  plan  for  a  program  of  research  on  this  topic.  It  is  not  our  purpose  to 
design  c3  systems,  but,  rather,  to  identify  researchable  human  factors  problems 
relating  to  the  performance  of  such  systems. 

As  one  means  to  this  end,  ve  are  convening  a  series  of  workshops  to  bring 
together  small  groups  of  appropriate  individuals  to  discuss  various  aspects 
of  the  problem.  One  such  meeting  is  scheduled  to  be  held  in  Cambridge,  Mass. 
(BBN  office  at  10  Moulton  Street)  on  October  6  and  7-  The  purpose  of  this 
meeting  will  be  to  discuss  man-computer  dialogue  development  in  relationship 
to  C^  systems.  In  particular,  what  we  hope  to  .accomplish  is  the  articulation 
of  some  significant  researchable  problems  pertaining  to  this  topic  in  c3 
contexts . 

This  memo  is  a  follow-up  to  the  invitation  you  recently  received  by  phone 
to  participate  in  this  meeting.  We  are  very  pleased  that  you  have  agreed  to 
meet  with  us  and  look  forward  to  seeing  you  on  the  6th.  We  have  limited  the 
number  of  participants  to  not  more  than  20  so  we  can  maintain  a  lively 
discussion  in  an  informal  atmosphere. 

What  we  want  at  the  meeting  is  a  far-ranging  discussion  of  issues  that 
participants  consider  to  be  germane  to  the  general  topic.  To  provide  a 
context  and  some  points  of  departure,  we  have  drafted  a  brief  background 
paper  on  C3  systems,  and  prepared  a  list  of  questions  that  are  representative 
of  those  that  we  might  discuss  at  the  meeting.  Both  of  these  documents  are 
enclosed. 

We  plan  to  devote  the  first  morning,  or  whatever  portion  of  it  is  required, 
to  brief  (10-15  minute)  presentations  by  participants.  While  the  invitation 
is  not  contingent  on  your  agreeing  to  make  such  a  presentation,  ve  would 
appreciate  it  very  much  if  you  would  be  willing  to  take  this  time  to  give 
your  perspective  on  the  general  problem.  If  you  wish,  you  might  use  the  time 
co  introduce  possible  topics  that  you  feel  are  especially  important  to  explore 
during  the  meeting. 

The  meeting  will  start  at  9'- 30  a.m.  ,  Wednesday,  October  6,  and  end  eai’ly 
in  the  afternoon  of  October  7.  If  you  would  like  us  to  make  hotel  reservations 
for  you,  please  indicate  that  on  the  enclosed  self-addressed  postcard  (specify 
the  night  or  nights),  and  return  it  to  us  as  soon  as  possible.  Enclosed  is  a 
map  of  the  area  to  help  you  find  your  way  to  BBW. 


MEMO:  Workshop  No.  6:  Man-Computer  Dialogue  Development 

From:  R.  W.  Pew 
16  September  1976 
page  2 


Probablv  the  easiest  way  to  Ret  here  from  the  airport  is  to  take  a cab. 
If  the  driver  does  not  know  where  Moulton  Street  is,  tell  him  it  u  in  the 
Fresh  Pond  area  of  Cambridee.  He  will,  of  course,  reimburse  you  for 
reasonable  expenses  Incurred  as  a  result  of  your  participatron  (travel, 
neals  lod*in<0  and  can  offer  you  an  nonorarnn  of  $150  per  day.  Fernapo 
the  more  significant  motivation  for  participation  is  the  chance  to  help 
stope  a  new“D.lBPA  program  that  is  likely  to  provide  research  opportunities 
for  human  factors  specialists  in  the  future. 


See  you  on  the  6th! 


Enclosures : 

Background  Notes 
List  of  Questions 
Self-Addressed  Stamped  Postcard 

Map 
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WORKSHOP  NO.  6:  man -computer  dialogue  development 


October  6  and  7,  1976 


We  would  like  to  focus  this  meeting  on  researchable  issues  in 
the  development  of  man-computer  dialogue  techniques  and  methods 
that  are  relevant  to  application  to  future  systems.  Since  there 
is  so  much  lore  and  so  little  data,  it  will  take  a  concerted  effort 
to  focus  on  issues  that  are  amenable  to  research  as  opposed  to 
discussing  the  relative  advantages  and  disadvantages  of  particular 
techniques.  The  following  incomplete  set  of  questions  is  designed 
to  stimulate  such  a  discussion.  Please  feel  free  to  add  to  this 
list. 

I .  METHODOLOGY 

1.  Given  that  one  lias  defined  an  application,  what  methods 
may  be  employed  to  systematically  select  and  design  the 
most  appropriate  techniques  or  procedures  for  man-computer 
interaction? 

2.  What  role  should  task  analysis  play  or  user  requirements 
analysis  play  in  defining  dialogue  specifications? 

3.  What  role  should  protocol  analysis  play  in  defining 
dialogue  specifications? 

4.  Is  it  possible  to  simulate  alternative  dialogue  forms 
prior  to  completion  of  applications  software? 

5.  What  role,  if  any,  should  formal  experiments  play  in 
the  development  of  dialogue  specifications? 
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6. 


Is  it  possible  to  conduct  formal  experiments  in  a  way 
that  the  results  will  lead  to  generic  principles  of 
dialogue  specification? 

7.  At  what  stage  of  system  development  is  it  most 

appropriate  to  initiate  the  development  of  dialogue 
specification. 

II.  TAXONOMY  OF  DIALOGUE  NEEDS 

1.  Is  it  possible  to  formulate  a  mutually  exclusive  and 
exhaustive  set  of  generic  classes  of  requirements  such 

as  numerical  data  entry,  inquiry  of  data  base  and  so  forth 
having  implications  for  design  of  system  "front-ends"? 

2.  If  we  had  such  a  taxonomy  would  it  be  useful? 

III.  MODELS  OF  THE  COMPUTER  USER 

It  is  often  asserted  that  dialogue  specification  would  be 
greatly  facilitated  if  we  could  develop  a  conceptual  model  of  the 
user  and  the  way  he  "thinks,"  about  the  problem  to  be  solved. 

1.  Is  it  realistic  to  think  that  such  a  model  might  be 
derived  either  in  general  or  in  specific  applications? 

2.  If  only  specific  applications  can  be  considered,  what 
are  the  conditions  required  that  might  make  such  models 
realizable? 

3.  What  form  might  such  models  take? 

4.  What  methods  might  be  employed  to  develop  such  models? 


[ 
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TV  dialogue  cost-effectiveness 

T  c  software  retire,  to  imP-cnt  various  dialogue  or. 

may  vary  fro.  si.pie,  ^stored,  branching  —teres  -  -ct 

artificial  ti.e  ana 

sophistication  has  ^  prooessing  system.  Methods 

on  operating  dema  the  impact  of  building  front-end  software 

are  needed  for  evaluating  the  impact  «  J  potential  payoff 

at  varying  levels  of  sophistication  m  terms 

for  overall  system  effectiveness. 

y.  „hat  measures  might  be  proposed  to  evaluate  the  economic 
impact  of  introducing  alternative  dialogue  techniques. 

2  what  measures  might  be  developed  to  assess  the 
complexity  of  software  required  to  implement  a  particu.  ar 

dialogue  concept. 

3  „ow  are  "ease  of  use"  and  data-entry  reliability 
evaluated  in  terms  of  the  impact  of  introducing  alternative 
levels  of  sophistication  in  front-end  design? 

*•  •<  «»■  ™. 

— a-,  e.™......  “  ••<*«“•  “■«” 

•  nf  ii  sers  do  we  believe  eve 

1.  what  characteristics  ol  users 

important? 

2.  How  do  we  assess  or  measure  the  levels  along  the 
dimensions  we  believe  are  important? 

3  how  do  we  translate  specifications  of  user  characteristics 
into  usable  constraints  on  dialogue  generation. 
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V. 


alternative  dialogue  implementations 

The  following  list  of  alternative  dialogue  forms  is  suggestxve 
of  the  range  of  techniques  available. 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 
9. 


Menu 

Form- filling  frame 
Question  and  answer 
Prompted  input 

Limited  syntax  command  language 
Typed  natural  language 
Real-time  interactive  speech 

Constrained  natural  language,  either  speech  or  typed 
Interactive  graphics 


one  can  think  of  many  possible  issues  concerning  the 
conditions  under  which  one  or  more  of  these  techniques  is  particular. 

useful.  Among  them  are: 

!.  HOW  does  the  Display  device,  for  example  printing  terminal 
vs.  CRT,  impact  on  the  choice  oi  technique. 

2  Dialogue  control  and  front  end  intelligence  may  reside  in 
the  terminal,  in  a  communications  concentrator  or  in  a 
host  computer.  How  does  this  location  impact  on  the  choice 

of  dialogue  forms? 

3  If  control  is  remote  what  is  the  impact  of  communication's 
bandwidth  on  choice  of  dialogue  implementation. 

4  What  is  the  impact  of  potential  system  response  time  and/or 
its  variability  on  choice  of  dialogue  techniques. 

What  are  the  implications  if  real-time  speech  input  is  to 
be  integrated  with  other  modes  of  interaction? 

is  it  possible  to  teach  users  to  operate  with  a  con¬ 
strained  vocabulary?  If  so,  what  are  the  limits  in  practical 
applications  and  what  are  the  training  implications. 


5. 


6. 


: 


Is  it  possible  for  a  user  to  learn  to  use  a  restricted 
set  of  grammatical  syntax  with  or  without  constraints 
imposed  on  vocabulary.  We  know  they  can  use  a  fixed 
syntax  and  unconstrained  syntax.  The  interesting  cases 
lie  in  between. 

What  does  the  discipline  of  psycholinguistics  have  to 
contribute  to  the  design  of  command  language  syntax? 


MEMORANDUM 


p  *1 


To:  Conference  Participants 


From:  R.  w.  Pew 


Subject: 


Notes  from  ARPA  Workshop  No.  6  on  Man-Computer 
Dialogue  Development 


Date:  7  January  1977 


In  connection  with  the  ARPA  sponsored  BBN  program  concerning 
Human  Factors  and  Command, Control  and  Communications  Systems,  a 
v/orkshop  was  held  on  October  6  and  7  under  the  general  title  of  Man- 
Computer  Dialogue  Development.  The  workshop  brought  together 
computer  scientists  and  psychologists  interested  in  issues  relating 
to  the  design  of  friendly  interactive  computer  systems.  No  attempts 
had  been  made  to  impose  an  organization  or  apply  any  judgment  to  the 
topics  that  had  been  discussed.  These  notes  simply  present  the  ideas 
in  roughly  the  same  order  as  they  were  presented  at  the  meeting. 


NOTES  from  October  6  and  7  Workshop  Concerning  Man— Computer 
Dialogue  Development. 


The  RITA  System 

The  session  began  with  a  description  of  the  Rand  Interactive 
Terminal  Agent  System  (RITA).  RITA  is  intended  to  be  a  component 
of  an  intelligent  terminal  development.  It  is  an  information  system 
in  which  "agents"  act  on  behalf  of  the  user  to  carry  out  relatively 
routine  and  prespecified  tasks.  It  is  most  useful  in  situations 
that  are  in  part  event-driven,  that  is,  situations  in  which  the 
need  for  activity  can  be  predicted  relatively  well  qiven  the  _  ’ 

development  of  certain  contingencies.  Thus  far,  it  has  been  applied 
to  a  problem  in  air  search  and  rescue  and  to  modelling  terrorists 

behavior . 

For  example,  in  search  and  rescue  activities,  the  user  might 
ask  for  the  status  of  the  aircraft  in  the  vicinity  of  an  accident. 
RITA  would  then  activate  an  agent  to:  (1)  define  those  locations 
.in  the  immediate  vicinity  of  the  accident,  (2)  send  messages  to 
those  sites  to  inquire  of  available  aircraft  status,  (3)  receive 
messages  back  from  those  sites  concerning  aircraft _ status ,  and,  (4) 
assemble  those  reports  into  a  single  report  communicating  aircraft 
status  back  to  the  user.  The  system  provides  an  explanatory 
capability  because  it  can  report  its  own  logic  chain  leading  y 

to  the  output  of  a  particular  response.  A  further  important 
feature  of  the  system  is  its  ability  to  accept  new  knowledge 
incrementally  and  to  introduce  new  rule  sets. _  A  question  was 
raised,  however,  concerning  exactly  how  easy  it  would  be  for  a 
novice  user  to  accomplish  such  changes. 


In  the  context  of  the  discussion  of  the  use  of _ a  RlTA-like 
system  to  systematically  reduce  the  message  processing  workload 
that  is  presented  lor  a  commander  to  review,  the  question  was 
raised  as  to  the  need  to  define  or  impute  a  purpose ^to  the  user. 
That  is,  in  choosina  what  messages  he  wishes  to  read,  the  designer 
must  relate  that  set  to  the  purpose  for  which  he  wants  to  read  _ 
them,  and  those  purposes  are  not  static.  Thus  it  v.’ould  be  diffi 
cult  to  build  such  a  system  to  meet  changing  needs  realistically. 


The  RITA  system  appears  also  to  provide  a  rich  environment 
in  which  to  examine  the  modes  in  which  user  requests  for  action 
can  be  most  effectively  communicated  to  a  computer. 
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Variable  Influencing  Design 


A  discussion  was  held  concerning  some  of  the  important 
dependent  and  independent  variables  of  interest  to  individuals 
attempting  to  design  effective  man-computer  dialogues.  The 
dependent  variables  include: 

1.  Amount  of  training. 

2.  Range  and  type  of  users. 

3.  Time  to  perform  tasks. 

4.  Error  tolerance. 

5.  User's  preference. 

6.  Creativeness  of  solution. 

Included  in  the  list  of  independent  variables  were: 

1.  Spoken  vs.  typed  commands. 

2.  Formal  command  language  organization. 

3.  Type  of  terminal. 

4.  Precision  of  information  presented. 

5.  Natural  language  vs.  formal  language;  written  vs.  spoken. 

This  outline  led  to  a  discussion  of  the  problem  of  errors. 

It  was  suggested  that  we  need  to  build  a  taxonomy  of  types  of 
errors  and  that  we  need  a  methodology  for  locating  in  advance 
potential,  error  prone  circumstances  in  a  dialogue.  We  do  not 
have  good  theories  of  the  deep  structure  of  errors  that  permit 
generalizations  from  one  application  to  another. 

Research  and  Application  Methodology 

The  question  of  methodology  breaks  into  two  issues.  The 
first  question  concerns  how  to  do  research  that  has  the  potential 
to  impact  on  future  system  designs.  The  second  concerns  the 
methodology  for  developing  actual  systems.  With  respect  to  the 
fitost  question,  one  could  consider  controlled  studies  in  a 
laboratory  or  naturalistic  observations  in  a  ■ ield  setting.  It 
j_3  likely  that  there  are  a  continuum  of  meth  between  these  two 
extremes.  It  was  observed  that  there  are  only  a  very  few 
laboratory  studies  to  date  that  have  been  conducted  in  a  mode 
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that  generalizes  to  future  systems  design.  It  is  not  clear  that 
we  really  know  how  to  do  laboratory  studies  of  this  kind. 

The  two  best  examples  of  laboratory  studies  are  of  those  of- 
Yntema  and  his  colleagues  concerning  the  effects  of  system 
response  time  on  performance  and  Chapanis'  studies  of  interactive 
communication  between  pairs  of  individuals  and  among  small  groups. 
Chapanis  described  his  experiments  involving  two-person  problem 
solving  teams  in  which  one  individual  was  given  a  specific 
problem  to  solve  and  the  second  individual  was  given  the  informa¬ 
tion  sources  needed  to  solve  it.  The  two  must  work  together  to 
produce  a  solution.  He  has  studied  a  variety  of  communication 
modes  ranging  from  face-to-face  with  full  speech  capabilities  to 
a  remotely-linked  interactive  typewriter  mode.  He  examined  the 
activities  that  people  undertake  in  solving  such  problems,  the 
variables  associated  with  their  communication,  the  quality  of 
their  solutions,  the  time  required  for  solutions,  and  their 
attitude  about  communications.  The  three  most  important  results 
were:(l)  while  10  t.o  15  times  as  many  words  are  used  in  a  speech 
communication  mode,  the  problems  were  solved  roughly  twice  as. 
fast  in  the  speech  modes  than  in  the  written  modes;  (2)  both 
spoken  and  written  dialogue  bore  virtually  no  relationship  to 
formal  English  grammar  and  structure  —  the  remarkable  thing  is 
that  people  can,  in  fact,  read  through  the  distortions  of  the 
English  language  that  are  introduced  in  communications  of  this 
type;  and  (3)  in  experiments  with  restricted  vocabularly,  it  was 
found  that  the  time  to  solve  the  problem  did  not  change  even 
when  the  vocabulary  was  restricted  to  300  words. 

With  respect  to  the  methodologies  for  systems  design,  four 
approaches  were  discussed:  protocol  analysis,  paper  and  pencil 
simulations,  computer- implemented  simulations,  and  "software 
overkill . " 

Protocol  analysis  involves  in-depth  interviews  with  represen¬ 
tatives  of  a  user  population  who  are  fully  knowledgeable  with  respect 
to  the  way  in  which  the  operation  could  be  computer-aided. 

The  first  goal  of  protocol  analyses  is  to  identify  the  collection 
of  strategies  that  a  user  can  employ  to  make  inferences  from 
available  information.  One  attempts  to  abstract  the  underlying 
structure  of  the  problem  from  the  interviews.  Protocol  analyses 
have  been  applied  over  a  range  of  levels  of  specificity  from 
generic  analyses  of  human  inference  structures  to  studies  of  the 
goal  structure  and  inquiry  patterns  associated  with  a  particular 
task.  It  was  pointed  out  that  if  one  is  working  in  a  practical 
context,  it  is  very  difficult  to  conduct  a  protocol  interview 
without  leading  the  user  to  report  what  you  as  the  interviewer 
expect  to  hear. 


Taper  and  pencil  simulation  is  a  natural  extension  of  protocol 
analyses.  In  this  case  an  inference  structure  and  dialogue 
protocol  are  developed  and  the  user  is  presented  with  a  series  of 
frames  representing  the  conduct  of  an  interactive  session  with  the 
computer.  One  can  solicit  information  concerning  vocabularly, 
conceptual  understanding,  and  the  appropriateness  of  the  branching 
structures,  but,  since  the  interaction  does  not  occur  in  real  time, 
it  is  difficult  to  use  it  for  evaluating  many  of  the  desirable 
characteristics  of  an  interactive  dialogue  that  result  from  its 
temporal  coherence. 

The  next  level  of  sophistication  in  interactive  dialogue 
evaluation  would  provide  a  computer  implementation  of  a  prestored 
dialogue  structure  and  sample  frames  as  they  would  appear  to  the 
user.  It  is  not  necessary  in  this  simulation  to  represent  the 
underlying  processing  structures,  only  the  frames  as  they  would 
appear  to  a  user.  Prestored  scenarios  would  be  introduced  that 
allow  a  user  to  progress  through  a  well-specified  activity.  The 
simulation  developer  would  anticipate  the  major  branciies  and  the 
kinds  of  data  that  would  be  needed  for  a  user  interaction.  This 
level  of  simulation  permits  evaluation  of  the  real  time  interactive 
features  of  the  dialogue,  but  again,  it  is  limited  to  the  de¬ 
signer's  conception  of  how  the  system  should  work.  It  requires 
iterative  application  of  the  method  to  evolve  new  strategies  of 
interaction.  The  assembled  group  could  think  of  no  specific 
examples  where  this  had  been  done  but  it  was  argued  that  it 
would  be  very  feasible  to  develop  a  general-purpose  dialogue 
simulation  language  to  make  it  easy  to  implement  alternative 
sequences  in  this  way. 

The  final  proposal  for  the  development  of  computer-based 
user  aic  for  novel  applications  involves  what  John  Brown  refers 
to  as  "ha.,  dware  and  software  overkill."  As  a  developmental  stage 
in  systems  design,  it  was  argued  that  a  very  general  purpose 
system  should  be  built  that  would  enable  very  easy  implementation 
of  many  alternative  configurations.  With  such  a  system  it  would 
be  possible  to  put  together  alternatives  in  a  few  days  that 
normally  would  take  several  weeks  or  months. .  While  the  initial 
investment  in  such  a  system  would  be  great,  it  would  make  it 
possible  to  explore  alternative  interactive  modes,  to  examine 
the  evolution  of  user  response  and  to  genuinely  adapt  the  system 
as  new  patterns  of  user  interaction  emerge. 


As  a  footnote  to  this  methodological  discussion  the  human 
factors  practictioners  in  the  group  pointed  out  that  simply 
exploiting  the  technology  for  developing  user  requirements  through 
task  analysis  and  protocol  analysis  and  proceeding  to  write  a 
user's  manual  prior  to  system  implementation  could  go  a  long  way 
toward  creating  a  friendly  system  with  no  further  technological 
developments  necessary. 

Role  of  Artificial  Intelligence 


Another  major  theme  of  the  workshop  concerned  the  role  of 
artificial  intelligence  in  the  development  of  "friendly"  systems. 
Ira  Goldstein  presented  an  articulate  summary  of  his  work  on 
kr owl edge  support  systems,  including  an  office  planning  assistant, 
and  an  intel] igent  advisor  for  the  WUMPUS  Game.  He  argued  that  an 
intelligent  assistant  for  either  case  implies  a  model  of  user 
knowledge  and  a  model  of  user  interactive  strategies.  In  any 
system  which  is  going  to  provide  intelligent  advice,  it  is 
important  not  only  to  provide  a  model  for  the  decision  process 
but  also  an  articulate  "expert"  that  can  explain  the  rationale 
underlying  the  decision  algorithms  invoked.  Similarly  the  model 
for  user  interactive  strategies  also  implies  an  articulate  "tutor" 
that  can  rationalize  the  strategies  that  have  been  implemented. . 
Effective  models  of  user  knowledge  and  interactive  strategies  will 
push  the  frontiers  of  artificial  intelligence.  The  intelligent 
advisor  to  WUMPUS  represents  a  level  of  system  complexity  for  which 
such  a  development  seems  feasible  at  the  present  time.  It  could 
serve  as  a  very  effective  laboratory  environment  in  which  to 
explore  human  factors'  questions  concerning  the  information  that  is 
needed,  and  the  ways  in  which  that  information  should  be  structured 
and  presented  to  human  users. 

The  discussion  went  on  to  describe  some  of  the  properties  of 
useful  user  models  to  imbed  in  a  friendly  system.  If  we  could 
achieve  a  genuine  understanding  of  the  user's  knowledge  base  and 
his  interactive  strategies,  several  modes  of  assistance  to  users 
would  be  possible.  It  would  be  feasible  to  offer  advice  concerning 
alternative  actions  to  take  and  to  provide  the  rationale  from  why 
those  are  appropriate  actions.  It  might  be  possible  to  begin  to 
go  beyond  simple  feedback  reporting  that  an  utterance  was  not 
understood  and  provide  feedback  about  what  the  user  could  have 
said  that  would  have  been  understood.  It  would  be  possible  to 
decide  when  the  user  needs  tutorial  assistance  and  when  it  is 
superfluous . 
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h_c  ImPlenienting  th^S(f  ideas  requires  development  of  the  knowledqe- 
base  concepts  underlying  them  beyond  the  point  they  are  currently 
developed  It  requires  a  theory  of  human  inference.  ?t  reqSires 
a  theory  of  hints  that  elaborates  how  much  knowledge  is  necessary 
before  a  hint  will  be  useful  to  an  individual.  It  requires  a  Y 
better  understanding  of  the  use  of  contextual  information.  The  art 
of  communication  is  based  on  the  fact  that  we  understand  each 
other's  world  view,  but  there  are  boundaries  on  where  context  is 
helpful  and  the  circumstances  under  which  it  is  a  burden.  People 
are  largely  unaware  of  their  own  ambiguities.  It  requires  the 
development  of  more  sophisticated  grammars  functioning  at  both 
semantic  and  pragmatic  levels. 

aoals6  nrn?iac;?ett?r  unde5standil?9  of  ways  to  represent  human 
goals.  Goal  structures  change  within  the  conduct  of  particular 

tasks  as  well  as  across  tasks.  We  cannot  treat  a  single  utility 
function  as  an  overriding  one  in  any  particular  context.  Goals 
provide  one  kind  of  context  and,  as  certain  kinds  of  knowledge 
become  available,  the  context  in  which  those  goals  were  relevant 
tend  to  shut  leading  to  a  changing  goal  hierarchy.  Analysis 
of  goal  structures  could  lead  to  broader  examination  of  the  rela¬ 
tion  of  between  human  motivation  and  system  design. 

The  development  of  "understanding"  depends  on  having  well- 
defined  and  limited  domain  boundaries.  Successful  systems  c«Uch 
as  SOPHIE  and  WUMPUS  depend  on  the  fact  that  the  domain  in  whSh 
inferences  must  be  made  can  be  well-defined  and  circumscribed 
Where  we  have  difficulty  is  where  we  attack  problems  of  inference 
ror  which  such  boundaries  are  difficult  to  conceptualize.  The 
limits  are  not  vocabulary  or  grammar  limits  per  sc,  but  rather 
limits  on  the  domain  of  discourse. 

A  further  underlying  theme  was  the  role  and  usefulness  of 
natural  language  communication  in  man-computer  dialogues.  Several 
participants  supported  the  belief  that  the  goal  of  natural 
anguage  communications  with  computers  is  an  essential  component 
f^1f!?dly  systems.  Setting  aside,  for  the  moment,  the  fact  that 
he  fuH  richness  of  natural  language  requires  a  model  of  the  user's 
world  view,  these  proponents  reject  the  argument  that  natural 
language  interaction  is  necessarily  inefficient  and  verbose.  They 
suggest  that  even  where  concise  and  brief  language  is  required 
the  model  of  natural  language  is  still  the  appropriate  one.  As  one 
u,  ta  point  m  support  of  this  view  it  was  pointed  out  that  the 

,+-C?l0ra?°1POlie0  f°rCG  inf°rmation  system  abandoned  the 
use  of  artificial  language  in  favor  of  natural  language  with  verv 
great  improvement  m  the  acceptance  of  the  system.  * 
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The  alternative  view  is  that  the  intrinsic  arabiguiti  - 
hllTnan  use  of  natural  language  particularly  spoken  language,  as 
well  as  its  inherent  verbosity  make  it  undesirable  for  interaction 
with  unforgiving  computer  systems.  Instead  we  should  be  seeking 
to  build  formal  command  languages  that  take  account  of  human 
stylistic  variables  and  that  impose  an  easy-to-use  formal  structure 
on inauiries  and  other  communications.  This  approach  seems 
particularly  appropriate  for  the  class  of  users  for  whom  the 
system  does  not  represent  a  discretionary  tool  but  rather 
integral  part  of  their  daily  activities. 


Graphical  Display  Capabilities 


Ncq re ponte  and  Bolt  described  their  work  on  novel  graphical 
display  environments,  and  illustrated  it  with  a  videotape 
demonstration.  They  think  in  terms  of  spatial  data  management 
as  opposed  to  symbolic  data  management  and  regard  a  display  surface 
as  simply  a  means  for  presenting  one  component  of  a  virtual  array 
o?  possible  displays.  They  wish  to  consider  the  possibility  of 
an  entire  room  serving  as  a  medium  for  display  interaction. 

On  the  input  side  they  consider  light  pens  and  eyboards  as 
t-oo  restrictive  and  are  experimenting  with  touch  sensitive 
surfaces  that  would  make  possible  the  use  of  gestures  as  a  mode  of 
communication.  They  are  working  toward  a  taxonomy  of  use  ux 
gestures  and  think  in  terms  of  a  language  built  up  out  o 
vocabulary  and  syntax  of  gestures. 

Touchinq  again  on  a  methodological  point,  it  was  argued 
atronalv  that  developments  such  as  these  highlight  the  importance 
of  nonconducting  formal  evaluations  of  the  capability  of  specific 
systems  but  rathL  that  design  must  proceed  in  evolutionary  steps 
because  the  understanding  of  the  new  way  in  which  systems  may 
TseTit  a  discovery  process  that  is  difficult  to  anticipate  in 
advance  The  use  of  raster  scan  TV  technology  to  produce 
relatively  inexpensive  color  displays  and  to  make  it  possible  to 
produce  camera-ready  copy  directly  from  the  display  were  cited 
examples  of  developments  for  which  it  would  be  hard  to  anticipate 

the  advantages  and  uses. 

I s s ues  in  the  Command  and  Control  Contex_t 

James  Hiller  presented  a  perspective  on  the  command  and 
control  context  in  which  interactive  systems  might  be  used.  He 
noted  tha?  commanders  exhibit  considerable  reluctance  to  use  a 
machine-based  decision  aids.  Some  will  even  refuse  to  log  on  to 
such  systems.  Me  need  to  investigate  the  effectiveness  of  system 
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personalization  and  to  demonstrate  the  ability  to  augment  the 
capabilities  of  a  commander  in  order  to  produce  acceptable  systems. 
A  commander  probably  needs  a  large  screen  display  capability  and 
a  conferencing  capability.  A  researchable  issue  concerns  the 
need  for  color  displays  at  this  level. 

He  noted  that  the  command  and  control  environment  is  time 
sensitive.  At  the  highest  level  of  WWMCCS  we  might 
have  thirty  minutes  warning ;  once  an  enemy  launch  has  been 
detected,  if  it  is  an  airborne  launch,  there  may  be  only  two 
minutes  of  time  in  which  to  act.  If  it  is  a  submarine  launch, 
then  there  may  be  fifteen  minutes  remaining  to  act. 

There  is  some  interest  in  considering  distributed  data  bases 
for  command  and  control  tasks;  however,  there  is  no  research 
concerning  the  advantages  or  disadvantages  to  the  user  resulting 
from  distributed  data  bases.  There  is  also  a  need  for  the 
development  of  standard  measures  of  command  and  control  systems 
effectiveness  that  could  be  applied  to  evaluate  the  advantages 
of  computer  augmentation. 

The  introduction  of  data  bases  that  are  accessible  at 
different  levels  in  the  command  raises  a  further  question 
concerning  the  capability  of  a  commander  to  second  guess  the 
activities  and  analyses  of  lower  echelons. 

The  ensuing  discussion  highlighted  the  great  need  for  more 
global  studies  of  the  organizational  impact  of  imposing  computer- 
assisted  decision  aids  and  information  systems  in  a  military 
context. 


